It sounds like you are handling things about as well as you can. I liked the way you described the interaction - answer his questions pleasantly but not in detail, and don't ask him anything, just validate. I would just say that he really shouldn't be dropping by the house unannounced if he isn't living there. My wife and I have a schedule for her to visit the dog, so that she won't just "accidentally" run into me.
In terms of him invading your thoughts, that will happen. It's hard but the trick is to acknowledge the thoughts, hold them lightly and let them pass. Ignoring them or burying them doesn't work, but neither does letting yourself get swept up in the emotions, and the cheeseless tunnels of repeated thoughts. It's a delicate balance that I am still struggling with myself. Journaling can also be good. Some days are going to be better than others, but you will find with time that the thoughts lessen. I also will focus on my breathing to bring myself back to the present moment.
Worrying about him having a R with someone else is a cheeseless tunnel. Right now your relationship is over, and you need to let him go. His actions are beyond your control and worrying about them does nothing to help your state of mind or your sitch.
Hang in there. It sounds like you have some good GAL options, so take advantage of them, and when you have the kids focus on them.
W 34 Me 42 Married 7 years together 8 0 kids 1 beloved dog BD 4/6/2018 I moved out 4/7/2018 I moved back in alone 8/05/2018 I file 3/06/2019 D official 5/7/2019