I don't know how old your son is. Sounds like he is old enough to see for himself what is really happening. I think you did well in not letting your H place that on you. That is HIS, he owns it.
Please be careful about speaking negatively toward your H in front of him however.
