I would simply tell your husband that he doesn't get to dictate to you how you communicate with your children, and that you will NEVER lie to them -- EVER -- but will always tell them the truth, in an age-appropriate way.
And the next time he tries to justify his affair, say something like "I will not debate with you the morals of having an affair. Your decisions and choices are yours alone, and they are yours to live with. Please don't ask me to make you feel better about your extremely hurtful decision to have an affair."
