MPT, I don't think I am like your H. Wish I was. I must be high need. Maybe I'm screwed up but no sex = no love to me. Sex would probably not be an issue or focal point if I was getting it frequently. Its like when you are starving all you can think about is getting some food. When you can get food often enough you don't focus on it anymore.
Sooner, hearing that you can relate to my thoughts helps me big time. For so long my W has been telling me I'm a freak for being obsessed about sex that I was starting to believe it. When I gave her chapter 1 of SSM I said, "See! I'm not crazy afterall!"
Sounds like you should be much softer in your approach than I have been. I was half out the door due to her EA so I held nothing back.
The most important thing a father can do for his children is to love their mother.