Sherry is right! This is not a natural thing and just as I did with my H and my H did with me, we say mean, cruel, and hateful things. To me this is a lame attempt to justify to ourselves what we have done. My H has gotten himself mixed up with a married woman himself and is doing no different than what was done to him by me. It doesn't pay to throw this up to him or say anything to him at all. You may as well throw another brick in the wall that's already tall and still considered under construction.
You worry about taking care of you and taking care of your children. Find happiness within yourself and the improvements and changes you make to you. This will be contagious to your H. I'm assuming you are both living under the same roof and this is an advantage when doing this. My H and I live apart and we only see each other for a little bit on Sundays and then only on occasion otherwise. Utilize this and let the wonderful things you find in yourself spread through out your home.
Do your darndest to not become bitter and allow it to consume the woman he loves. This is very hard to reverse. In my research I have learned that men are passionate overcomers and women are passionate caregivers. It has nothing to do with being weak, it's just the way we are made. Continue to be passionate about caring! I've come to love my life and I'm very happy other than the fact that H is missing in action. I've learned that the world keeps turning.
It won't be easy as Sherry said, but it does get easier! There are a lot of wise people here who are going through what you're going through. If you need to vent take it out in prayer if you are a believer. Just don't do it in front of your H or your kids! It takes practice to do this, but make sure you do everything you can to smile as much as possible. Remember, it's contagious.