How much do you get to see your kids? I think you need to get visitation rights, don't wait until they grow up and the kids ask you why you didn't.
It doesn't have to be a major fight, unless there is abuse or neglect of somekind if you are their father a judge should give you some visitation. This may upset your XW, so what, she wanted the D. Just remain calm and don't get into a word war with her. emm you have rights to your children.