Well, to him you are punishing him because he's missing that closeness/connection with you when you pulled back.
Whats better vocabulary to use, from "punishing someone for their behaviour", to "protecting yourself because they are hurting you"?
Supposedly, "boundaries" are meant for this kind of discussion. but sometimes, even a word like "boundary" can be taken as too strongly. it's tricky, depending on how sensitive the person is that you are talking to.
My current status: june 2006. Wife ran out and filed D. Finalized Jan 11, 2010, after 12.5 years M. 3 wonderful sons caught in the middle