I'd be more prone to losing my love from the years of extreme neglect, myself.
Combine the two, and it'd be a slam dunk. Having your cake and eating it too is one thing... I'd most likely be willing to forgive that. (I think... it hasn't happened to me yet, so I can't say with certainty) But to spend years indignantly refusing to share any cake at all with me and offering excuse after excuse as to why she's unable to bake, and then go on a baking frenzy and still refuse to share any cake with me while bringing every last morsel to someone else, that strikes me as one of the ultimate in dealbreakers.
Eddie, this is EXACTLY what I have the most trouble with. This is not just a once-recently-great marriage that's been temporary derailed by an affair. This is a marriage that for MOST of its 22 years, did VERY LITTLE to meet my most primary needs, emotionally and physically. I have 3x more hesitation and doubt about that than I do with my ability to heal and overcome the affair.
For now, I've determined that I can see enough things done wrong, by me and Mrs. Choc., that there's a chance of good success if we can get back together and try to do things better and differently, so I press on. That patience and forgiveness, though, will not last forever, and I have told my wife this.
Sadly, she rushes away from me faster and more determined than ever. God help us.