I think my H's the same. He cut off any physical contact quite abruptly. When I later learned about the OW, it made sense that he had started seeing her at that point. My coach says I should try to get him to cheat on her with me. Kind of odd, but I'm going to run with it.
You cannot teach a man anything; you can only help him discover it in himself. Galileo Galilei