"It DOES seem to be a regular convo for him to remind me of how hard his life is & how much simpler & more fun my life is (I have heard it for 20 years!!)."
if he's been saying it for 20 years, then it means NOTHING.
its a technique to fill a break in the conversation. Some people talk about the weather, he says his life is hard.
i'd guess he says the same thing to the RE girl, and the dog walker and the guy at the gas station, and the mailman.
just because someone talks, doesnt mean they're saying anything. he's not reminding, he's not comparing, he's just filling dead air. stop forcing it have some deeper meaning. and stop thinking it has anything to do with you.
"In a ham and eggs breakfast, the hen is involved, but the pig is committed".
