Can you get into a IC appointment - I'm guessing that you probably need to talk to someone.
Do you think you would want to call or meet up with him for coffee? I can't help but think that he said the damage can't be repaired - does he mean damage to you or to him?
You've done a lot to become a better person, married to him or not. So you haven't wasted your time. I think you also deserve more for your efforts. I'm not saying chase him down, but getting a fair and clear answer that is a bit more than "can't repair" is important.
Now if it is depression speaking, that might be him giving up.
Another thought - maybe a MC session with his current IC? Maybe the other IC could (in session) help you figure out if he needs time or his walk in the storm to see you meant nothing.
Twice in the last 3 wks I was sure there was no hope, despite suddenly feeling love for my W, because of her comments or reactions. I was sure that we couldn't "repair" the past. More than twice, I've felt that we could. Who knows what he's really thinking, but I hope your patience and love for him can get you through this really low spot to wherever your future lays.