Finally, your excuses in regard to your son rub me the wrong way. I have one child. My daughter. She is in college now. It would require a prison cell, or walls reaching to the heavens, or a military army to prevent me from being a big part of my daughters. 3 hours. Autism. House fires. None of that would keep me from being a father to my child.
I have that same gut reaction. Bunches, you've done a lot right--being the main caregiver for your son for so many years, passing him on to his mom when you got distracted by life circumstances. I have an ex family friend whose child is special needs and I get it's hard. My son had special needs for one year. I do wonder if W were not in the picture you'd have made different choices after the "mysterious house fire" (I smell smoke and a story!). E.g., moving closer to your XW so you could co-parent as a team while you figured out a workable living situation where you and son and W and W's sons lived together, or spending weekends in town with him. From the outside, it *looks* like a symbolic moment when you lost a bit of your self, prioritizing your W more, shifting more into co-dependency.
Figuring out what you'd need to have a bigger role in your son's life again may be exactly what you need to find your self anchor and get your mojo back. Besides, I bet he misses you.