I'd have to ask...what does your gut tell you? My friend stayed in her M long enough till her heart, mind and soul gave way to sheer exhaustion. In the end, she left her now XH for her own sanity and happiness.
When the man you love constantly accuses you of cheating when there is nothing going on, you'd get pretty tired, too.
In her case, though, she left the M early enough so that her child wouldn't grow up to be a helpless witness to their struggles.
By the sound of it, you sound almost as tired as she was...