He is trying to come towards you. Don't make it harder to achieve than it already is (in his struggle).
Sounds like this is your approach and it is a good one. Don't feel bad about it. It is a very unselfish thing we do if we do it correctly.