LL,
When your children are young they cannot comprehend "all of the truth and nothing but the truth",
and you cannot be sure of what/how they will understand.

I would tell them that Daddy did not come home from work. Your son's thinking that Dad lives at the office says it all from a 3 yr old's world.
This is what your son can process and he is already questioning his importance to his father.

Do not protect your H - ALWAYS PROTECT YOUR BABIES - and yours are truly babies.
As children get older (hopefully this will be behind you) in a few years you can tell them
that Daddy is/was (depending on the outcome) having a difficult time and he needs us
to be his friends, love him and pray for him.

Of course, in a few years you may well be a mended family, but H will still have to answer to his son.
Kids don't forget. They do forgive.
You are doing a wonderful job in leading your children by example.

Faith. Hope. Love.
LSL