Your response was fine. He gave you information regarding caring for the baby next weekend and you acknowledged receiving the information. It was good to keep it short and to the point. It didn't warrant any more response than that. Of course you are going to wonder where he is going, but he doesn't need to know that. You are going to need to keep communicating with him regarding the baby, but keep it as informational and business-like as possible. Re-read Sandi's rules until they become like 2nd nature in your interactions with him.
He might very well be going out with OW, but you are catastrophizing when you constantly imagine the worst case scenario. As hard as it is right now, you need to let it go. You know that you can't control him, but you still want to. You want him to come over and interact with you and take the baby to his mother's, because that feels like it might be a step in the right direction. But it isn't up to you and what you want. He is going to do what suits him and at this point he is acting like a selfish jerk so his decisions are not often align with what you want. You have to let him go and make his mistakes and continue on his journey. At some point he may wake up and realize the mistakes he has been making but that is less likely to happen if you don't let him go.
Hang in there. You have a lot of support on here.
W 34 Me 42 Married 7 years together 8 0 kids 1 beloved dog BD 4/6/2018 I moved out 4/7/2018 I moved back in alone 8/05/2018 I file 3/06/2019 D official 5/7/2019