Hope, From what I've read, they need to "reconnect" periodically with the things or people that were a part of their past life, a life that was essentially the happiest time of their lives. It's not the guilt that has them reconnecting, but desperation for sinking further and further into depression. When they have those moments of clarity (as Sting referenced a long time ago), they need that contact, to know that the life and/or the person that they once knew is still out there. To someone who hasn't been in the depths of the dark depression, it is extremely difficult to understand.
All you can do is be you. If you have a conversation w/him, talk to him just as you would a neighbor or a co-worker that you haven't seen in a long time. That's all he needs to survive a bit long until the depression grabs him again.
The anger is normal process for us. As we begin to heal and think about what they've done, we have anger surface and it helps us to process and do what is necessary to go the next step in moving on w/our lives. I experienced that as well and it pushed me to the next level to continue moving further away from the fallout of my xh's crisis. The ow will pull many different things to suck your h back into their drama. The baby ordeal was awful for you, but you've got to find a way to let it go. The people in crisis become very irrresponsible during that time. They don't think pregnancy, addiction, health problems will happen to them. The euphoria is what is causing them to fly so high, just like bipolar individuals have that "high" and the the big let down. This is what is happening to those in crisis. Their highs are very high and the lows are extremely low once they hit that all time dark depression. Our coaster ride is bad, but their's is even worse, if you can imagine that. At least we can find things to get away from them and their behavior, but they are stuck with those demons in their heads, hearts and souls 24/7. Now, I'm not making excuses, I'm just trying to explain what I've read about, observed and spoken to those who have experienced a crisis.
Definitely speak up if he raises the subject. He needs to be held accountable for what he's done. He's far enough along now to be able to hear and try to process some of what he's done.
Yes, my darling little devil has remarried. He married to ow and has been married about 4 years. I seriously doubt that the new bride knows what he's done and is continuing to do. I sometimes wonder if some of the hang up calls are from her looking for him. I do know that he doesn't put her name in cards and doesn't reference her in conversations with his aunts and uncles. They also have different telephone numbers in their respective names at their residence. Sounds like roommates, doesn't it? Sad, but it's a true example of "be careful for what you ask for".
Hope, you aren't alone in this. There are a lot of posters who are experiencing exactly what you are going through. It's tough and it takes time to find a place where you are comfortable in dealing with his behavior. But, I can say this, you will get to that place when you are ready. It took me a very long time to figure out what I needed to do to get there myself.
Hope, here is the book that I read that really opened my eyes as to what transpires in people w/depression. It is called "Unholy Ghosts, Writers on Depression" by Nell Casey. It gives you a different perspective as to how people in depression can be.