If you remain in the relationship with secrets, you have shown him it's OK. So the other option is to walk away. And then that's it, you don't talk to him for a while. Let him stew in the loss of you. Let him feel horrible. You're scared to do this, I'm sure.
I'm sure he also has a trick up his sleeve. Take away his power to use those tricks. You are capable.
If he is mad, that's probably good. He is stomping his feet about not getting his way. Boo hoo.
He's yelling at other people to leave you alone, but he won't treat you right? Boo hoo. When you say he's your EX, what is the relationship status?
H 34 W 29 BD 3/12/18 Divorce Busted Spring 19
It is not things that bother us, but the stories we tell ourselves about things.