I've so been there. Your sitch reminds me of mine a few years ago.
My H has been home for a few years now. And I will tell you that those feelings of anxiety do subside. For a while, though, you may feel uneasy and that's okay. Don't talk about your M. Don't talk about the A. Don't ask him what's on his mind, but do let him know you're there for him if he wants to talk.
Remember that breaking off an A is very confusing and difficult for the cheating spouse as well. Recognize that. YOu are not the only one who is afraid. He probably is afraid, too.
That said, do keep the lines of communication open. Find ways to connect again. Are there activities that you two used to do that you can start doing again? Maybe it's finding common TV shows. Maybe it's going golfing or bowling. Maybe it's taking walks. whatever it is, figure out something that the two of you can have together.
If you do find him treating you in a way that is not acceptable, do not tolerate it. Stand your ground and be strong.
Piecing is difficult. There is a balance between being a safe haven for your spouse and being strong enough to stand your ground.
You can do this. And remember, it's okay to be scared. The piecing process takes a long time, but it is most certainly doable.
Married 9 years Kids 5 and 6 Bomb 2006 H back and forth for a year M now back on track