He's saying personal things about you, but it's truly not personal. It's not about you. If what you've told us so far is true, he's angry and scared and it's easiest to blame you. He's not doing it because of any failing in you but because he can't stand to examine his problems. Look at what he's saying; he's not really saying that what he's doing is OK, but he says he wants you to give up on him. Hard to say what he really wants, but that's the fear talking.
Do NOT back down from your determination to go see a therapist who can help you with this. And do NOT doubt that you have every right to want and ask for as much sex as you want. He doesn't have to give it to you, but you're not obligated to pretend that sex isn't important to you. He wants that (or thinks he does) because sex makes him look at his problems, not necessarily because he doesn't want you.
