You and your husband are literally, soooo....close to fixing this, yet soooo....far.
I think if you each had your cup full of humor and joy, the interactions would improve.
Really they would.
Instead of focusing on what you are not getting from your husband, think about what you are getting.
Try this. For 2 weeks...writer down 3-5 things you are grateful for that day. Do this before you go to sleep. Make sure you write it down. It can be rather mundane: my husband picked up the kids and made things easier for me that day, the apricot jam on my toast, the sunset, my husband placing his hands on my shoulder. Do it every night for 2 weeks. You overall mood and PMA will increase. You'll smile more.
And as you PMA goes up, you might actually do that get a life thing. You might even get your groove back. And then you'll be oozing grace, and joy and self-confidence (rather, Christ-confidence). And guess what? You WON'T be oozing what you aren't getting from your husband. And that will make you very attractive. He'll want to touch you and ravish you.
On your husband's side. He's got to stop kicking himself in the ass about his mortal wound -- the affair. Once he's done that, he'll actually be able to get a good job. That will help him get his groove back. And then he'll want to ravish you, and you'll want to support him and love him.
In some sense, we all love to kick people when they are down. Don't know why, but it seems we do. You are both down, so you are both taking turns kicking each other.
How about this, instead of saying, "Stop kicking each other", which is logical, let me say, "Just get up." If you get up, he'll stop kicking you. If he gets up, you'll stop kicking him.
How that for trying something different? By the way, it's elementary Divorce Busting.
BI -- Get up.
It's what I've been trying to help your husband do.