I think many parents feel this way when going through a S or D - I can bear the pain for myself, but really struggle with the pain caused to my children.
Others have already given good advice. Own your anger and your pain and process and get it out in the best way you can - away from your H.
When it comes to his H with his child/children - try to act from your highest self and facilitate that contact when he wants to have it. And protect your own heart in that process. There is no need for you to spend time with H if that doesn't work for you.
Your H's life is his to live as he best sees fit - as is yours. And I think you are doing a darn good job of it in difficult circumstances.
Take care Cherry xx
T 13 M 7 Me 48 H 46 SS 15 BD 7.14 PA D final 5.16 (H filed)
We receive & we lose, and must try to achieve gratitude & embrace with whole hearts whatever of life that remains after the losses - Dubus