I really know the feeling. I am sort of in the same situation here. While I have some good friends in this area of NC, I am painfully aware that I no longer have any serious ties here -- other than my two S's. xW, xMIL and their family have soured me on this place, and all of my NC relations have long ago passed away or moved away. And there are too many negative memories that I am still trying to overcome, it seems. If it were not for my two kids, I would have pulled up stakes a long time ago.
Coincidentally, I have relatives still in eastern TN, as well as AR and MS. So I have been thinking for some time, since the S, that western TN (Memphis eastward, as far as Nashville) might be a good central location, reachable to/from all family.
But I am looking at another 13 years at least before I can think about that, after my youngest has flown the nest.
I do understand and empathize with your sentiments.