You can support her church involvement without going yourself by picking out the things that are good about it and smiling and showing your happiness about those. You don't even have to accept it if she makes commitments that she can't keep, or commitments that obligate the family to do things the rest of you can't do or don't want to do. You just look for the things that you and she both think are good about her role in the church and show her your happiness about those things. It doesn't have to be big, overt gestures, either. The grumbling you mention as a problem probably wasn't composed of grand gestures of disapproval, right? It was probably offhand comments and body language. Yet it had an impact. Positive but small, offhand comments will build up the same way.
I haven't been around for months . . . . have you sat down with your wife lately and told her that you feel lousy because you get the impression that she thinks her affair was justified, although regrettable? If so, what did she say? If not, I think it's likely that nothing else will help much until you clear that air.