It is great that you and H had a good breakfast outing. You need to view H as a friend, not H. This was a nice get-together with a friend. When you get together with your friends for lunch/dinner, do you worry about their feelings or reactions? No. Same with H.
Just had a good day with H so that is a small victory in your book.
