I gotta call BS on that one! Most of the pilots I know are not all that wealthy, in fact most are solidly middle class. Instead, they've made compromises in their lives in order to be able to fly (me included). That means not buying new cars, living perhaps in a more modest house, forgoing the latest electronics and the annual vacations to resort destinations. An aircraft can be co-owned among several partners to decrease each one's share of the expenses, or you can join a flying club or rent. Sure, some of the expense will take away from other things you could do with the family, but then what doesn't. Everything is a compromise. If you want it bad enough, you can get it. It isn't nearly as far out of reach as you indicate, especially if you settle on going the Light Sport Aircraft route. Sure, it isn't fast pointy airplanes, but it beats sitting on your couch making airplane noises pretending you are flying by a long shot. Heck, my 14 year old son is working odd jobs to get the money together to get his ticket, and he's already got about half of what he needs. He's also gathering the info and skills to build a plane from plans. If he can do it, I'm sure you can too.....if you want it bad enough.
It is all about how much you want it and what you are willing to sacrifice to get it. If you want it, do what it takes to get it. If not, then don't carry on that it is not accessible. If I understand you correctly, you already have your PP certificate from years ago. If so, then the cost for you would be a couple hours of instruction to brush up with and either renting or getting into a flying club. If you've already got a ticket, you could polish up your skills and buy into a club for very little money. So CAC, what are you going to do?