WH, I agree w/AJ...try to the best of your ability to ignore his accusations. You are the only one that knows the truth and you are the only one that can hold your head up high and look in the mirror and know that you have been honest throughout all of this. If you sit quietly, eventually people will see the real him and come to understand that the line of bs that he's been feeding them is just that...bs.
I know this is very difficult, but you need to step back and allow his fire to burn out. You do not want to add any more fuel to it for it to continue. When he realizes that you aren't taking this ugly bait, he just may stop some of his nonsense.
Sit quietly, the answers will reveal themselves when you least expect them to. The past is gone, the present is a gift and you need to focus on today, allow the future to reveal itself when it is ready.