What I am going to say will probably not make sense.
You must detach from the whole sich. After doing that you must look in the mirror at yourself and figure out what YOU can change about YOU.
Stop looking at him and the AP. Odds are that the thing that is appealing about the AP is that she was available and willing.
Your husband has some BLAME in this. He is not firing on all cylinders, however you must not worry about that because you can not change HIM.
All you can do is make yourself into a person that only a fool would not want. First you must love yourself enough to do that. To be able to find what is wrong may or may not be what your husband sees. Right now he is in an AFFAIR FOG and whether you can trust that judgement or not is highly doubtful.
SO TRUST YOURSELF FIRST.
So out of the things that your husband said. Are you FUN?