I believe that he has done those things to you. I also believe that you had a very volatile relationship with him and you yourself probably got out of hand at times. Your descriptions mirror relationships of several people I have known in my life.
Let me be clear, NOTHING YOU DID EXCUSES HIM HITTING YOU, SHOVING YOU OR ANYTHING ELSE.
BUT when a woman, ANY WOMAN, gets in a mans face I firmly believe he has the right to push her aside if that is what it takes to get himself the clearance to walk away. I know what I am talking about. My mother did that crap to my stepdad when I was a teenager and I saw him "move" her on many occasions and I would not classify those times as "abuse". HOWEVER, there were other times, towards the end of their relationship...where he clearly did get abusive....so I know the difference and don't lightly excuse EITHER of you.
The fact is Emily, you have a lot to learn and you haven't even begun to understand that.