SRTTF, sorry that things are not going well. I know you do not want to rock the boat but you need to protect yourself. Your H is in a fog and not your real H, he is not thinking about your needs or how much you need him. He is dual addicted to the drinking and the OW (like my H), you can't save him but only yourself till (if) he finds his way home.
Get legal advice, get financial support. Your H needs to be accountable for his family like it or not.
take care
Me 53 H 50 D16, D29 M 22 years bomb 7/08 INH - alien pod replaces H 8/08 - OW (direct report), I work there also bomb II - H moves 10/1 expose ow 10/22 D to be final 9-09