NM - I would venture to say many - if not most - WASs try to save face by telling the OW they "have" to go home. What's their alternative, really? Admit that all the stuff he fed her was lies, lies, lies? Tell her he realized he really does love the LBS and the affair was all a huge mistake?
He is probably hedging his bets with you in the same way - saying he "has" to come home - but don't be fooled. He's just saving face. You haven't been pressuring him at all lately - in fact, notice that when you dropped the rope and started to seriously consider moving that he suddenly "had" to come home?
He's coming home because he WANTS to be with you, no matter what else he says.