>But then the family, wife, kids didn't really directly participate in the crime so how and when to punish them?

Pretty sure being the recipient or beneficiary to stolen or embezzled funds makes them an accomplice.