For months after BD if H had told me he was angry because I wasn't licking the floor enough, I would have been down on my hands and knees licking the floor.
At some point we all need to realize that we are not the problem. And work on ourselves just because it is a productive thing to do while our lives are in turmoil. Not because we created the turmoil, not because it will solve the turmoil, but because it is the healthiest way for us to survive the turmoil. But it is easy to go a little too far into the "self blame" mindset. I promise you, every single married person on earth has some dysfunction and something that they have done that annoys/upsets/creates a reaction in their spouse. Our spouses were no picnic pre-BD if we are honest with ourselves. So the question isn't who is to blame, or how much you are to blame, or how much you hate yourself or feel guilty. The question is what are you going to do about it, how are YOU going to survive in the healthiest, most graceful, dignified manner?
I mean it from my heart when I say you are both kind and committed and compassionate people. Don't beat yourselves up.