I'm sorry that happened. I know we are not supposed to believe what they say but you do need to listen to the way he is talking to you. I think he is still being abusive to you and that it is not healthy for you to be around him. I am not saying your relationship cannot work I just think you need to get away from him for awhile for your own mental health. He is being very hostile and mean to you and whether he means to be or not the fact is that it is verbal abuse.
I know how much you want this to work and how much you want to save your marriage and how hard you have tried and are trying but where does it end with taking his abuse. He is using you as a verbal punching bag. In my honest opinion I think you need to get out of the house and be by yourself for awhile. I just can't see how staying with him, when he is acting this way, is going to help your case and it is certainly not going to help you mentally and emotionally.
Take care of yourself first ITH. You are infinatly more important that your marriage.