There's no telling how long his affair will last. It has to die a natural death and one that you didn't help along. Why? Because if you interfere w/that relationship, he will only step up to the plate and protect her for all she's worth and you will come off looking like the mean witch from Oz. When the affairs die a natural death, it will because one or both of them have called it quits because it's not working out. That could be a month, year or even years down the road. Most of the time, MLCers tend to stick w/one ow. Some do experiment w/others along the way until the find one that will meet their emotional needs. Just keep in mind that if he hadn't hooked up w/this one, he would have found someone else.
The ow is nothing more than a crutch to his crisis. He's on an emotional journey to figure things out as to why he was hurt so long ago. You may not even be aware of childhood issues/drama because he's suppressed it for so many years. It could have been something that he blocked out and didn't want to discuss...but when midlife rolls around, aging begins, etc., that's when something flips the switch and they begin to look at where they are and that life is getting far too short and they have to go back and attempt a "do over" to see what they think that they've missed.
If you have apologized about the possibility of calling him a pervert, then stop doing it. Once or even twice is enough of an apology. I suspect he's now projecting on to you what he thinks he thought he might be...a pervert. So, the next time he brings up the subject, just tell him you are sorry that he feels the way he does and that you've already apologized to him about the comment. Then find something else to do or walk away, but don't stand there and allow him to make you feel bad about what transpired a while ago.
He's trying to find a reason to leave home and you've not given him one. As for the IC...many of them visit with the IC and then state that they are doing it to help them leave the marriage. They hear what they want to hear and run w/it.
I hope that you are reading some of the other threads on this forum. You will discover that your h is going through some of the same things that others have posted here. You aren't alone...we are a family. I would also suggest that you read as much as you can about MLC and depression. Knowledge is power.
Please try to keep the focus on you and your family as much as possible. I know it's difficult, but you've got to take care of yourself and be sure to watch your finances and bank/credit card accounts. It's important that you do because the norm is that they do love to spend their money and yours as well.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.
