I do believe that OW has something to do with it. It doesn't have to be any pressure from her - but he very likely is comparing the old and the new. And honestly, it's hard to compete. Not until she's old too!

He keeps asking of you the opposite of what you are giving. There is no way to win here - not right now. He is not thinking rationally, he's being tugged from two sides.

You need to give him space, if he wants sex, I guess give him that too. But I think at this point, I'd sit down and have a talk with him about him letting you know what HIS needs are. Not a deep R talk, but just letting him know that if he wants help with things, he should let you know, etc. That is what I'd do. It is preferable to going nuts.