What jumps out at me is that your H said he didn't expect to hear from you, that you never call him, he always calls you.
You can almost hear the hurt in his voice. Have you thought about ever picking up the phone and just calling to say hi? Maybe you should give this a try. If he spews at you when you do it, remind him of what he said.
Maybe he feels like you don't love him anymore only because he can't understand it, feeling as low as he does about himself.
Most of us really marry only once. First love endures, even unto our dying day. And we never really divorce.