I also need help in LRT with kids. I still find that we have far too much interaction. He often wants to continue talking with me. How was your trip? Who did you meet? Was it interesting? etc etc. I try to be vague. But it is such a departure from how we were its hard to seem so "rude".
Why are you worried about if you are being rude if he is having an affair? That is kind of rude too.
Maybe this is meeting some kind of need of his? Best to stop meeting his needs while an affair is going on.