Ooh...I am so with you there about having unwanted OW thoughts pop up.
If he truly wants to work on things, get yourselves to counselling. write lists of what you expect in a relationship. write lists of deal breakers. Have him to do the same. Read books...bugger...gotta go and amuse the child...stay strong...at least he is home and the affair is over (isn't it?).