Does he have any kind of schedule that he is currently keeping with regards to visiting S? How is money currently handled?
In my case, My H said he would come to visit D's one day a week (and has kept to that) and we agreed to a $ amount (verbally) that he has always had to me on time.
This has just been my experience and believe me I know how very lucky I am. Would I hesitate to address it if I needed to? No.
What do you think you need to do? If you need to have a legal agreement to protect yourself, then that's what you have to do. Whether you're poking him or not.