You're getting great advice here. There's a few things that could be contributing to the distancing. I've heard and read a lot about a grieving time after the end of an A...even bad relationships need to be grieved to some degree.
He may have been very depressed and using OW to hide from his pain for awhile. Now that is over and he may still be depressed and now having to feel like crap for what he has done too.
Cave time is needed. My H had a relatively short A, but took 9 months to come back. I think the cave time helped in Jeanine's husband's case also, as I recall.
You're doing great. One thing you may suggest is that while he's living in his own place--you two date. Really do it like your dating. It helps in a lot of ways, and is also a comfortable way to transition.