You are doing the wrong thing, at least how u are describing it. He told you, you always stayed on the pc and not with him. He wants your attention and to protect yourself you are pushing away. It's ok stay calm I'm here with you.
Remember to stay calm make him glad for coming. You are standoffish and I understand this you have to protect your heart but the way you describe it he is making an attempt let him. Maybe he is the one that needs the validating.