Hope, most kids react like your son. Dont go into a antagonistic trip now. It's a trap. You are his mom. There is no "who is better parent" fight here.
Think about it: would you have felt better if your son came back terrified and didnt want to go to his dad again?