If it were me, I'd certainly be angry when he kicked me to the back burner like that. Anger is a completely reasonable response IMHO.
The "I let myself feel" stuff works better when both of you are making an effort to be open and communicative. I feel that he is testing you in some way, almost seeing what he can get away with. And I feel that you're fed up with his behavior and are seeking to convict him (not saying that is inappropriate on your part at all).
