I know how it kills the ego. But you just have to remember this isn't about you, not at all. it is all about him. He is confused and scared and ping-ponging all over the place. Honestly it makes absolute sense that after a day of being super close he backs away. You can't let it get to you.
You've come a long way. Go back and look at your first posts, or your early journals, or whatever... there are so many improvements. Maybe the biggest next thing H needs to understand is that his fantasy of unmarried best friends with benefits is just that, a fantasy. You might consider closing up the muffin shop again, double down on GALing, being flirty and positive when you're together but mysterious. (I am quite bad at being mysterious. But I know I read some stories on your thread where you came back late and he was clearly wondering where you'd been. so I think you can pull the mysterious vibe off. Also, I like the idea of a confident, sexy, mysterious wayfinder strolling out the door looking hot and H thinking F**K.)
Me (46) H (42) M:14 T:18, D9 & D11 4/19 - 12/19: series of escalating BDs 9/20 - present: R and piecing