I don't necessarily think small talk is bad. As long as it's not initiated, pursuing or about relationship you are fine. He is seeing you passionate about a sport and as a friendly person.
I think I mentioned my DB coach says it's necessary to have a friendship at the very least it's better for the kids.. I see what she is saying, but I also had so much anger at husband for leaving, I didn't wNt him to think we could be friends. Also I took his invitations to go on family trips as more then he did.