I agree that you should drop it now that the message has been left.
I know this is really hard. I think you had an expectation of spending the evening with him. It takes time and alot of effort to get those expectations down (preferably to zero), but it will help YOU. Question: Does he not want to be around you (telegraphing your anger/hurt) b/c of you or him (guilt/shame)? I know I ask alot of questions (and if you find them annoying please let me know), my purpose is to get to (and honestly, myself) to look at things from a different angle. Truth be told you don't know what he's thinking or how much he's hurting. You mentioned conflict-avoidant,boy have I got some experience there. Are you aware of some of the issues that tend to go along with conflict avoidance? If not, it's something to consider.
I hope the rest of the evening goes better for you.
