From my gut Nero I say "IF" you can go there and understand that this man is not your BF, not anything more than himself who's life your not a part of, send him off to his secrets without flinching, than go! But, can you really do that? Like you said when you had it all, how do you go backwards and be friends while watching him give what was yours to someone else!
I, would stay in NJ, and start what will be the first day of my new life! Spend summer, the best time to be out and meet people, around my own town, in my own home, so when winter hits, maybe you will be more secure than the last. It is time for reality to become our reality!
IMHO! I, we, will support you either way, we get it!
It's time he gets it too! Your not a yo-yo! IF he wants to visit NJ, let your roommate come and go about his business. I think even if this is not what's in your heart, it will show him you have moved on! He has moved on! Find Nero's path!
I would make separate bedrooms, and pick ups at the airport would be asked..not expected, you may be busy that day!
Let me know what you think!
The past can't be ahead of you in the future. You don't have to figure it all out, just pick a direction. What's next...I don't know but I can't wait!