Don't beat yourself up because like what has been said is the fact that you are dealing with a really hard situation.
On the other hand if you ask him to move back in after you have set a clear boundary would be a HUGE mistake. I mean HUGE. But that is just my opinion. But once you set a boundary like I am not going to live like this because he is having an affair then you ask him to move. Well, to take that back shows a. you don't respect yourself. b. you look desperate and c. you lose all credibility if a boundary ever gets made again. You see he will think that if he with woth her you may get mad and ask him to move a couple days but then he can be nice and you will giove in.
I don't want to seem mean faith. But you seem very clingy with him. Like wondering what he is doing and wanting him to only be with you. I think a great 180 for you to do is to go out there and really GAL. Let him wonder what you are doing and where you are. He is so use to you chasing. Let him do the chase.
Also I think you mentioned that your husband is having a PA. If he is kissing her and mot making love to you I honestlyu think it is a PA. But again that is just my opinion. Just be careful honey. The biggest thing that scares the hack out of me with affairs is the partner contracting some disease. : ( Then bringing it to an innocent person. This just happened to a very close friend of mine.
But just try to live life for you right now. Find what makes you happy again.