I forgot one thing.. I am going back on this specific line. " stopping him from talking to his kids he would have been fighting me all the time to see them".
I am not so sure. His words would have fought you but his actions would have showed you differently. My Ex would cry to the children about how much he missed them and wished he could see them everyday yet, he had excuses to not follow up with our visitation agreement. He was not so happy when my son chose to move in with him for a year. He pushed son away MANY TIMES. Many fights between the 2 of them. Many trips for me to be there for my son. MANY! It broke my heart but they needed to live it. Son moved back here after the school year was over.
They see their father but their anchorage is me.