Don't let your H's harsh words affect your self-esteem. You are a good person and you know it.
Your H's reaction is pretty bizarre. I suspect that his bitter hostile disgust about your sexual history is really projection. There is something about himself that he cannot face that disgusts him completely, likely related to sex but perhaps not. If I had to guess, I'd guess that he has a heavy internet and/or phone sex addiction OR (given his very rigid judging moral outlook) that he is gay and can't live with himself OR that he is lost somewhere in a huge mess of lies related to finances.
He is clearly over the edge and something within himself is putting him there. You making out with someone 20+ years ago is not the problem.
You are right that the problem isn't OW, it never was. But, the problem isn't you either. Don't let him redirect his inner hatred of himself onto you. It won't help anyone, it will only hurt. He needs to find whatever it is in himself that he hates and confront it head on.
You need to take about a million times more space than you have already so that he cannot keep blaming his problems on you.
