The only person Piano you have to be worried about is the innocent child born into teh situation and you being responsible parent doing what is best for that child, not your h, not your family, just your child.
He has 50% parenting rights J-star. I am afraid I could not deny him if I even wanted.