OK - just to play devil's advocate here - high school is really difficult these days, and going back and forth between houses must make it twice as hard to keep track of school assignments etc.
Since he's 14 now, is he interested in living with YOU? Surely if he wanted to, the courts would allow it at this age. Are you in a position to have him M-F and supervise his schoolwork? Sometimes a teenage boy needs to live with his dad.