Hey, sometimes our spouses *are* complete *sses. And sometimes *we* are. Was your ultimatum about sex?
The only way I know to get past an earlier past failure is to build a newer history of actions that counteract it. You were abandoned while pregnant 20 years ago. In the relatively short amount of time you've been with your spouse, he has had a dalliance of some sort with one of your family members right before you were married. He has participated in some type of internet flirt/chat sort of thing just a couple of years ago. He makes unilateral decisions without your wholehearted agreement.
These things don't instill trust.
Ultimatums are usually best when they're very specific. A generic "things have got to get better" is the equivalent of nailing down jello. This is where lists come in. What specific things will have to change? And you both get to put together the list. And I'm talking specific lists such as:
1. Sex once a week. Wife initiates 1 week, hubby initiates next week.
2. No unilateral economic decisions - moving, job, toys, trips, etc. without *both* partners agreeing. That doesn't mean one partner deciding and then brow-beating the other into submission.
The above is just an example, you would have to decide which issues you want to emphasize. I would keep the list short and to the point.
If you keep sensing that you are playing the martyr role, look to see what you can do to change that. Even if it means examining your past and seeing what you could have done differently, so that you'll be more aware of it in your present. While he left you without financial support, unless I'm missing some key piece of information you could have claimed child support and he would have legally had to contribute to the children he helped create.
Do you resent him for going on and having a more "exciting" life than you had?