This certainly wont replace being a navy/AF pilot, but have you ever Looked into flying ultralights? No license is needed.
When Richard Bach (a flyer thru and thru) was grounded for several years because of finances? ... I think it was... he found his refuge when ultralights came about a few years later.
I totally understand valuing a dream and having a desire more valuable then life. Ever see the movie 'Gattaca'?
