I know what you mean! I have the same thoughts sometimes, just don't know what to do or what to say.
I think it's good that you went back to doing what you were doing. If you dropped everything when he comes, then he might not feel comfortable and would see it as pursuing.
Maybe try to "guess" what he's up to. If he's reading a book, then he probably doesn't want to talk and would like to be left alone. If he looks like he's open to conversation or something else, then try to suggest something that you know he'd like. It sounds like you were doing that though a few days ago...watching the YouTube video he was watching or helping him out when he was working on the truck.
As hard as it seems, don't stop doing what you were already doing. If he seems to start thinking that he's not included, that's good...it will make him more curious.
