I fully agree with DnJ. I too have 3 children and fully understand the position you are in.
I am also a hopeful person and have always followed the path of approaching my husband in a calm, forgiving and friendly manner, without actually being a doormat. If he said or did things that were hurtful to me or the children, I would say in a direct but gentle way to stop it and then walk away from him.
This helped in most cases.
If you feel that he should somehow be present at the party, then you should go ahead with it. You have to let your own feelings speak.
In most cases, when they are surrounded by people other than the family, they tend to act more calmly. Let's hope he does this if you still decide to invite him.