hhh, you did nothing wrong. H way overreacted and you are right not to respond now. Just wait until he calls on Sunday but know beforehand what you are willing to agree with (which I think you already have).
Hugs!
If you love somebody, set them free. http://tinyurl.com/2empx2g