You don't have to go to dinner--its up to you but if its makes things easier for you for him to bring food for the kids then I say yes
you can always excuse yourself run an errand while he is there ask if it would be ok to leave and take time for you so even if you don't feel like his presence maybe the kids can be with him and you can disappear for a while
married 14 years H 42 bomb 2/07 IDLYA D final 3 /09 M ow D ow