It is awfully confusing isn't it? Because on the one hand maybe he's not feeling like coming back, maybe he's just temp checking and seeing that you're still there for him. But I agree that it must be hard for them to ditch the OW and be alone in the hope that you'll have them back. That's a big risk, to be alone, so at some stage if he wants to come back then he will creep towards you and you don't want to be too rejecting that he feels like there's no chance.
I have no advice, I'm CADG remember, but I just want you to be careful, I'm sure you are thinking hard about all this and won't keep your expectations too high xx