I think sometimes this is OW triggered, sometimes it is just because H is so confused and guilty and YOU are the person he can take out his anger/guilty/feeling of indecision on.

Take care