Just a little insight for you on getting over the OW...it takes TIME!...my H was still "in love" with the OW for a good year after they totally broke things off...it took another year before he could tell me he love me again...
While it seems they changed over night when they exited...in fact they didn't...it took time for them to develope feelings and to loose feelings for you...which means a reverse of that needs to take place...they need to "unlove" their OW and start to love you again...it all needs time...it has been 12 weeks...at least he is talking positively with you...hold on to that...I am not sure making him leave now would be the best option...but that is my opinion based on how my H was when he returned...