He may mean that it isn't like that. I had a friend who I was close with, and one day he kissed me. Well it wasn't like that. For me anyway. It never happened again. Sometimes the moment takes you away and that is all it is. So I don't really know what to tell you about that.
There is more to the whole situation than an affair. Don't freak if he doesn't contact you right away. He knows you are mad and upset and he may give you a little time to cool of so to speak.
Don't apologize for your feelings, it is just time to start processing them. So that you can move forward where ever that is.
If you focus on the past, you ruin the future. You can only live for today.