I think this is a good sign. He didn't act hurt b/c he wanted you to go. Think about it. We men still have that little boy inside us that pouts/gets feelings hurt when we don't get what we WANT.
You did the right thing by leaving. Make him miss you. Sounds like he will. Again, time is on your side.
