I agree that you should leave him be for now. You don't know what's in his head, so stop second guessing. It's crazy making. Any time there is a closeness on any level be ready for them to avoid contact for awhile.
I don't see the cuddling as a bad thing either. WCW is right that the mood swings will sneak up on you, so you have to be ready.
Don't feel bad about your decision to let him come over, but use it as a guideline as to whether you want to possibly go there again. If you con't you get to make that choice.