His anger toward you might be stemming from his guilt about the affair (look up the stages of MLC). Sounds like he might be going through anger/replay, and that could take a long time to process, so buckle in for a potentially long ride.
My H would get angry and lash out about crazy nonsense, and I took it personally until I realized he's just trying to paint a negative picture of me in his mind so that he can justify his actions and leaving me. So now I'm focused on being the best person I can be, GAL, 180s, whether he's around or not. That's all you can do, and you do it for yourself, because there's no guarantee that anything we do will actually make a difference in the long run. And you'll feel better about yourself in the short run (and the long run too).
M: 43 H: 39 D: 14 Married 15 Together 16 BD: 6/2014 S: 8/2014 OW revealed 10/2014 Instigated dissolution 12/2014, in progress So over it!