I'm guessing you are in the UK so hopefully you'll understand my northern bluntness.
You may think he has a lack of self esteem but he doesn't build his self esteem by going out tupping other women. And then tells you to start dancing harder by initiating sex. He must be going round thinking he's a right billy big bo**ocks. Look at these women laying themselves down for me.
Enough.
I don't care what you've done in your M to think that you deserve this. Nobody deserves this level of deceit and mind games.
Get yourself out of the circus.
So what if he twists things. He'll twist anything at the moment. Let him blame you. So what. Your health and sanity is at risk.
How is this dynamic arising? When is he back in the house? Why is he back in the house? If he has to be back in the house then why do you have to be there?
