h - I can't beleive you are still thinking about this - it's a joke.
me - I'm sorry I don't mean it that way


H has a lot to learn about validating, doesn't he?

Don't be sorry. Validate him instead ("I can understand why you'd think that.")

and add your boundary ("It may not totally reflect what's happened here, but there are things in it I find it helpful to read. It's what I need for myself right now.")

it has been on my dresser/night stand for 1 week. I guess he just noticed it. Now, I feel bad and feel we have taken a backwards step.

Yep, guess he just noticed it. H's reaction had a negative effect on your emotions. Practice detachment so you don't get triggered that way by his statements. I don't see it as a step backwards. I see it as H imposing his thinking on you. You're the one experiencing yourself and you're the one who knows what you need, not him.