Is he taking S to doctor? Why would the three of you have to go together? Is this for you?
I am wondering if maybe you need to not be around for a visit or have something else to do when he asks to do something. Not in an unfriendly way, but more in an "I have other plans, maybe another time" kind of way. To help you get off of the 'coaster.