It sounds so much like my H's touch and goes. This is the time where you really have to muster all the strength and patience you can find. I remember Snodderly telling me the same thing when I went through it.
If he leaves again try to let it go. I found in my sitch if I made a big deal of it, it would make it worse. Only you know what you need to do.
