While I understand why you hold your tongue and don't speak your mind (cause it's a pain and you don't want to deal with his reactions)....that HONESTLY was one of the biggest keys to my H and I getting where we are now.
I stopped avoiding his reactions....he in turn was forced to deal with ME, as I am. I told him straight up one day, "If I've got something on my mind or something I need to get out of my system....I'm going to do it. You don't have to like what I have to say, but I'm not going to hold it in any longer."
The dynamic you two have whre he dodges and you get discouraged (when you speak your mind) WILL happen, but it's something that (when you are ready) you will have to ride out. This happened with my H and I too....however blowing his hidden activity out of the water was the catalyst to me saying...NO MORE! I decided at that point....he was going to hear what I had to say, and that was that. If he wanted to be a big baby and sulk, or avoid me....then fine. He did do that for a while when I started speaking my mind too.....but eventually he stopped. I had to stop avoiding his childish behavior though and do what I had to do anyway (which was not hold stuff in.) When I stated things on my mind, I didn't attack him....I didn't accuse either. I'd just say what was on my mind....eventually, he started responding and we'd talk things out. It did take some time for him to realize though that it was easier to talk to me......than avoid me.