Tell him that I prefer to talk about this face to face but if it is too uncomfortable for him to meet me, I will assume the adjournment is important for him and that I will let my L know that I agree to it.
IDK what the heck to do...
This would let you stay dark/dim. If he still wanted to talk to you after that he would have to keep working for it.