Charming...does he insult you outright like this often?
Yes he does but as you mentioned below it is to get the intended "fired up" response. He then tells me I need to relax.
I found something interesting in "After the Affair", edited for third party, originally written in second person:
----------------------------
"Certainly, while [he's] having the affair, and even after it's revelaled, [he] may feel more anger toward [his] partner than guilt. That's because the two emotions are inversely related. The greater [his] anger, the less [his] guilt; the more venom [he] directs at [his] partner, the less [he] directs at [himself]."
"The anger [he] experinece[s] may be a legitimate response to the way [you have treated him], or defense against the guilt [he] feels for what [he's] done - it is human nature to hate those whom you have ingured - wrote Tacitus almost 2000 years ago. What makes it hard to know the difference is that anger always feel sjustified, its a basic character istic of the emotion."
-----------------------------
In reference to his baiting games and insults...Spring (ATA) suggests this is masked guilt. I will let you decide if this is the case in your home. I think it is the case in mine.