You cannot allow him to physically assalt you. Arguing and getting upset is one thing but if it is getting physical you need to put an end to it. Him blaming you for his actions is just wrong. None of us make another person do anything.
Pick yourself up and end all contact with him except for interactions with your kids. I would for the time being stop CC. Apparently it is not working. I would go to C alone and work on healing yourself.
Take care, I am really worried if he is physically pushing you. That is wrong on so many levels.