I think the right thing to do is to talk to my current L first, regardless of what I end up deciding.
But after I tell him what I am not comfortable with, what am I supposed to say? "so, I am changing the L?" Or "do you think you can address these issues?" Or "can I speak to your boss to see if he can assign me a different L?"
Also, I am doing my financial disclosure statement right now. If I am going to change L, I want to give it to the new one instead of giving to current one and end up forwarding it to the new one, which would cost more. So say I tell my L what I am not happy with, and then suppose L says he would change that, but I do not think I can really offer him a chance to "improve" his behaviors, b/s I haven't even filed the disclosure statement and there is nothing else going on other than that.