You handled yourself with dignity, gave him yet another opportunity to step up to the plate and fix this, and set a healthy boundary. Good job.
I know this is really difficult, knowing that his brain is all messed up. I had the same kinds of issues with my ex (multiple concussions in his case) which made it hard for me to let go, thinking that he wasn't completely in his right mind. But I eventually came to see that, concussions or no, he was competent enough to make the decision to end our marriage, and it hadn't really changed him from who he had been in the past - just accentuated the bad parts.
Once I accepted the fact that I couldn't change the course we were on - I embraced change and my new life. Once I did that, my life got better. Not walking on eggshells is a marvelous feeling! Having lovers who cherish us for who we really are is fantastic!
You did your best, you can sleep well at night knowing you didn't abandon him - for five long hard years you hung in there. Lyme disease or no, this behavior towards you is just unacceptable. Stick to the high road, work out an equitable settlement, and stop putting your own life on hold.