Well, I am not IMP... but.... Do you want him there? I get the feeling the added stress might mess up the whole thing you are hoping to have with the kids. SO that's point 1.
Is the might be able to a pretty easy might, or a pain in the neck. If it is hard, the stress goes up. see point 1.
But, your first though was to invite him. First thoughts are often good thoughts.
So, I fear, I can't give good advice, other than to think about what you think will work bet for you and the kids.
