I think you are 100% right on your 180's. Especially about the part of doing nothing. My, you are such a whirlwind of energy, it must be tough for you to do nothing!
My one fear, however, is that the more you GAL and continue to involve yourself in outside activities, the more the marriage may suffer. This is married-singles behavior. So, I would just say to keep an eye on it. Try it. See how he reacts. I think it is interesting that your H had an affair right after your child was born. I think he has trouble not being the center of your world. By giving more of your attention outside the home, that may make him progressively unhappier. Or perhaps not. Just be aware.