Kech, you need space from him for your own sanity. All your actions are still about him. You have to stay out of your house when he is around for the baby, this will help you keep interactions minimal and send a message through your actions. Until he cleans up his act he cannot just drop in to play dad as and when he pleases. Set up a clear schedule and stick to it. Get out while he is around if he lingers , go watch a movie or go to the gym or may be even finish up all your errands and grocery shopping. Just stay out of his way for now, the lesser the interactions fewer the mistakes. A year down the lane both of you are going to have severe regrets, you talk about it and R or you divorce. Keep your list short, automatically it makes his longer.