If you do not want the D, then don't act like you are a willing participant. Let him know that you are against the D; but if HE files you will not stop him. He needs to know that you are standing for your M; if indeed you are.
Your vacillations are normal for the early stages of your sitch; don't give up now. I understand that patience can be hard to come by; believe me, I rushed back into reconcilitaion in April and that rush is one of the things that helped doom my marriage. continue to give him space, set boundaries as needed, validate, be friendly (but not a friend, he's with OW); and most of all, go out and GAL.
Like they say around here, this is a marathon; not a sprint.