Beacuse when it doesn't help you, it sets back your emotional state, whether you are seeing it or not. It means that the next time you interact with him, you might not be in as good a place as you could have been. Or, when you don't expect it, one of these little things could lead you to doing something you shouldn't. (Say you see a pick, and before you even realize it you have the phone in your hand.)
The point Jack was making (I think, not being Jack!) is that anything you do that isn't helping you, is hurting you. And anything that is hurting you, is hurting your chances of a R with him, and actually, with anyone else. And it is hurting your R with yourself.
