LL, When your children are young they cannot comprehend "all of the truth and nothing but the truth", and you cannot be sure of what/how they will understand.
I would tell them that Daddy did not come home from work. Your son's thinking that Dad lives at the office says it all from a 3 yr old's world. This is what your son can process and he is already questioning his importance to his father.
Do not protect your H - ALWAYS PROTECT YOUR BABIES - and yours are truly babies. As children get older (hopefully this will be behind you) in a few years you can tell them that Daddy is/was (depending on the outcome) having a difficult time and he needs us to be his friends, love him and pray for him.
Of course, in a few years you may well be a mended family, but H will still have to answer to his son. Kids don't forget. They do forgive. You are doing a wonderful job in leading your children by example.