Thank you, everyone. I have stopped texting and as expected he was bombarding me with replies.
I have ignored him successfully until he again asked 'what are your plans?.'
I replied "I have made my plans clear in my last message" referring to what my obgyn said. He replied "you do realise that it is going to mean it is very unlikely I will ever be allowed to help with anything again ever?"
Note that he needs to 'be allowed.'

I really do not feel like answering to him, but I think about something along the lines "I understand your concern, however I cannot come back and I am unwilling to risk the life of our baby."

I think I can just call him out on his (practically) threat by saying "so you are willing to be dictated on how you see your child and be a part of the lives of your baby."
He is a scientist and it should be clear as day that the fact and the risk are clear enough that he should not be pushing me. I am really disappointed of his behaviour but I am not sure what I can do now. I do not want to come back and risk getting covid (and going against my Obgyn) nor do I want to deal with his stupidity.