NO! You're not wrong! Your son needs you, too! Moms are crucial but dad's are too! HE NEEDS YOU!!! Don't, don't, don't! Your son will adapt!! I promise! Yes, it [censored] and it's a huge reminder of all you're facing every day but it doesn't go away if you are in his life less. He needs that male example and influence. Plus, it's good for you, too!
Start finding things that you and your S6 do together on the first night he's with you. My daughter has Asperger's so I understand the transition problems and lack of consistency. It will take time but if your time with him is structured and predictable, it will give him a sense of calm and he'll associate that with you.
