Girl, you have got to get out of your head. How do you know he wasn't "cold" because he had a headache? Or his stomach hurt? Or he just would have rather stayed in bed all day with you? You can NOT know what he's thinking and I advise you not to even bother asking.
You ML. AWESOME!
Seriously, are you the person who finds $20 bucks in the parking lot then stresses about it? Maybe I should have yelled to let someone know they lost the $20? Maybe I should go put it back where I found it?
You are making yourself crazy....I know you think it's him, but it's not. It's you. YOU are making YOU crazy.
Enjoy the good moments. Expect the "backlash". When that happens I imagine they are thinking "Wait a minute....I enjoyed that. I really did and I enjoyed it with my wife? But how can that be? I thought I wanted out? But if I wanted out, why was that so nice and comfortable? Why did I enjoy that? Oh, I know....I'll show her! How dare she make me like her? How dare she do something enjoyable with me? I'm going to be an ass now! I'll show her!"
Seriously. Let him stew, not you. You had a enjoyable morning with him and you get to. You deserve it. Stay positive.