I will even allow her to take him after she gets off work if I do not have anything planned with him.
You see why you need everything in writing? See how he wrote, "I will even allow" and "if I do not have anything planned with him?" Those with custody seem to talk as though time with the kids is a gift they are giving the non-custodial parent. You need all the visitation you can get IN WRITING. How long before those that are "allowing" suddenly aren't doing so and then there is nothing legal that says you can have visitation. Get overnight visitation every other weekend (Friday night through monday morning) and at least overnight one or two nights a week. Don't leave it up to the good graces of your spouse or only when they have nothing planned. Your spouse can very easily say, "we have plans" whenever she's pissed off.
In the end, it's not the years in your life that count. It's the life in your years. Abraham Lincoln
It is hard to fail, but it is worse never to have tried to succeed. Theodore Roosevelt