Bummedout, I'm so sorry to see you're in this spot, having recently just entered the same sitch myself. I don't know what to say about "others'" opinions entering into the relationship except that they're all BS. Your new relationship, I believe, needs to talk with you, not everyone else.
I'm dealing with the same thing right now, as my H has made it clear he's talked with everyone at his office (most of whom I've never met) about me, and apparently they all agree I'm horrible. (His words, "Everyone at work is mad at you.")
I don't pretend to have any answers, as my own life is in turmoil enough right now. But I have to believe that the "everyone says" argument isn't a fair fight. This is between you and your partner.
I get the whole job/finance thing, too. It can be a double-edged sword, for guys or women. There's no question the financial stress we've faced these past few years has contributed to the current situation. But blame-placing is no way to resolve the situation -- it's just an easy out, I think.
H 42 Me 47 DS 7 T 18 M 16 Bomb: 4/20/10 H leaves to live at Mom's: 4/30/20
My sitch: http://www.divorcebusting.com/forums/ubbthreads.php?ubb=showflat&Number=1990503#Post1990503