Do you have any idea what he *wants* you to say in these moments? I get the impression he is like a toddler who is pushing you..pushing you..trying to get a feel of where the boundary lies.
When you ignore him, he cranks it up even further.

Is he looking for permission to quit?

What is it that he wants from you in that moment?

Hugs to you, btw.