In the long term, I feel it is better to wait till he is ready, rather than him feeling pulled back into the R (regardless of whether you pushed or not). My H moved out "to be alone" only turning out to be with OW for a few weeks. I found out and told him "don't come back unless you end all contact with OW". Few weeks later he came back but he did not end the ocntact completely. He did say he had to come back or "he may not come back". I was too scared to stand my ground and let him. Now contact with OW is still there (no PA, not seeing each other, just email/phone). I feel that if I had stood my ground, he would have ended it sooner. Now I have a bad feeling that he will forever have the thoughts of OW in his mind because he was not ready.
One thing I learned through this is that we are all human. We all have our "faults". H can say everything is your fault but you know (or will have to learn to know) that it is never completely your fault or his fault. And at the end, it actually does not matter that much whose fault it is. You are here, and you are who you are. What you can do is everyday try to be a better person than the day before. A more confident, happier, more considerate person, learning and growing everyday. If he wants you back, that's great. If not, then you have become a better person anyway.