I think (gulp) this is one of the times that you are just going to have to not say anything, because you are right. If you fought it, he would just do it again. The hardest thing for me lately is to tell myself I canNOT control H at all. I can give my suggestions/demands, but he still has the final say. Always. I have tried to just let go, but its not easy.
I know what you mean about the 'fine line' of DB'ing and being walked on. I think this is a DB moment for sure. Act as if you don't care, don't mention it, and he'll be totally surprised.
Another thing I tell myself is if there is a chance that H and I get back together, there will be a time and place for all these things to be straightened out.