Yes, it is sad that as you pull off the GAL thing and detach, you do lose love for a wayward spouse if they don't respond right away and make a move back toward the marriage. However, as you communicate this (and you SHOULD communicate this), it is a very powerful attraction to them.
"He seems to be doing just fine without me, and is more confident than I've ever seen him. AND he says he still loves me, AND he says that he can forgive me! But he also says that his patience isn't limitless, and that everyday he's losing some more love for me. Maybe I'm blowing this here!"