Stop doing what you think is the "right"thing to do. Do not text him back.
If he confronts you again about not answering just say you were in the middle of something. It's not uncommon for some to be busy and not be able to anwer a call.
Did he leave a message?
If not, then how important was his call?
Bomb 8/09. Brief piecing 12/10. D-2/12 Two incredible kids D9,S6 Leading new life! “Success is not to be pursued; it is to be attracted by the person we become."