so you are not alone in this and i actually posted this question awhile ago. i don't know how to link to the thread but the jist of it is guilt.
My W does the same thing where we will have a nice night (like last night at Fireworks) and then the very next day she will call me at work (not be able to get in touch with me because of crappy T-Mobile cell coverage) and leave a message asking me if i am out to lunch with a female and she hopes it is worth it.
Sorry to interject my own story here - but i really think they see that they are having fun and it confuses them - "why if i can have fun and enjoy this persons company am i doing this" when there is no rational answer they either invent one or get [censored]#y about it (my case - at lunch with boss talking work - boss is 58 year old male - not what she was thinking)
keep your head and PMA up and your expectations down - stay strong, from reading your posts you are worth the wait - he will see it eventually - just look for the signs that he has realized it...
