You really should stop engaging him and don't let him bait you. When you get into these bouts with him he feels justified in walking away and calling you all those things. You feel badly because you're mad at yourself for acting that way. There's a part of you that feels that to a certain extent you deserve it. Everything you've written has been written before. We all knows how this feel.
The thing is if you don't engage, and keep it all business, then what is he going to say? If the shed is a huge issue to you and you don't feel comfortable talking to him face to face, then either spell it out in an e-mail, or let your lawyer handle it for you.
Above all don't let him bully you and push you around!
