detachment doesn't mean having limited or no contact with them.

Detachment is an emotional thing whereby you don't let their actions (or inaction) affect your emotional stability.

You don't get down if they don't call. You don't get happy if they do. if they act like a jerk you blow it off and your mood is not affected.

So to answer your question, you CAN detach even if he's being a jerk.

BFM

There is only one person who could ever make you happy, and that person is you.
David Burns, Intimate Connections