Sounds like the only issue you really have is that your name is on the mortgage, correct?
Do you have any other entanglements with him?
If he doesn't pay the mortgage then obviously it will go into foreclosure and assuming there is a loss on the books you will get a credit ding (7-10 years is a long time).
If you aren't planning to move or finance any major consumer debt like a car, then it probably isn't the worst thing in the world.
So what has he hired the lawyer to do? Come after you for 1/2 the mortgage payments or the down payment? Unless you signed some kind of promissory note to him then I don't imagine he has any legal recourse regarding the downpayment.
Interesting situation, but not one that seems to be much that you can do anything about.