If he feels trapped, he's free to leave. Just tell him "I'm not going anywhere, I haven't done anything wrong."