I like what Gypsy said "At this point, it's not about him, it's about you." She also made some very good comments on how/when to know whether the answer is yes or no.
You have to go w/how you feel, regardless of how it impacts your children. I know that may sound callous, but ultimately, your children want to see you happy again and they'll understand if that means you have to be w/out stbx in order to be the sunshine.