Well maybe his anger is one of the things you tell him you need him to work on for you. It took 14 years for me to tell my H I would no longer take anymore of his abuse. He is now going to anger control management classes.
A brain with all the old memories and resentment deleted so we can start again with a clean slate
Gosh if it could be this easy. To bad we have to do it the hard way and forgive and except and let go of all these bad feelings.
Anyhow good luck with the conversation it is a good idea. Your H may get defensive be prepared. Don't let it become a screaming match if he starts getting upset. Just simply tell him you need to tell him how you feel so you can work through it and get past it. If he does not want to listen to how you feel you cannot promise anything will change for either of you. But also remember if he tells you how he feels do not become defensive listen and respect it as how he feels. There are no right or wrong in feelings. We may have been lead to feel the way we do for the wrong reasons but the feeling is not wrong it was the way we percieved something that may have been wrong or misguided.