NM, it's very possible he doesn't mention you at all to the OW...I am no expert on that but wouldn't he be trying to keep you in separate boxes as much as possible.
I think treat your WH however you want to. You are too hard on yourself. I know you love him, but I reckon you've 'done' your share of nice! Being angry every once in a while (or 10 times a day in my case!!) can be good for the soul - but don't ruin it by feeling guilty afterwards though!!
Having said that, yes, acting peacefully with those who have harmed us is untimately good... but give yourself time to get there.