I've been reading your thread for a long time. You have taught me so much about loving someone, about supporting them even when they are hurting themselves (and hurting you too). My instinct has always been to just walk away when the going gets rough. So I guess I am answering your message to say, I don't have any good advice, but you sound a little down and frustrated, and I want you to know I am cheering for you. You have made so much progress.
Your husband's anger is probably not at you or at what you've done in the past, frankly I think he is just angry at himself, and can't believe you go on loving him and supporting him, and so he turns that anger at himself for being such a fool toward you.
He has obviouly told the OW things about your relationship that aren't true -- part of his fantasy or his fog -- and he knows it. So when he has to "defend" to her why you are at family functions, he has to lie all over again, and so he has to shift the blame to you. Think about it -- he says to you "too bad for OW" if she doesn't like something, and you two connect over it; you can only assume that of course he says things about you (that aren't true) to keep his fantasy with the OW going. But I think the anger is because the fantasy is crumbling and he knows it. He knows that one day this naive OW is going to wake up to his fence sitting, and he won't be the shining star in her eyes anymore. Meanwhile, you've been the woman who stood by him. And that will make all the difference.