Here is how I see it. I could be 1000% wrong and way off base. I am merely speaking as an observer.
Disclaimer: I am not trying to be unkind or come down on you. I am just going to spell out the realities as I see it.
You may have been DB'ing for a while but IMO you are DB'ing in a way that makes things very easy for your H.
Your son is little and it is simply too much for him to be exchanged in a parking lot close to bedtime so you accommodate your H by allowing him to come to the house. You are the primary caretaker of your son. That is what your H has chosen. Therefore your H's visiting hours should end earlier so you CAN pick him up in a parking lot at a reasonable hour. If there is no other way to handle it then your H should not be allowed to come in, chat and linger at all.
If your H is unhappy about having less time with his son or having to bring him back earlier, well, tough. This is what your H as chosen.
As your son grows up his needs will change (bedtime, social time, school time and so on) and that means the time your H gets with your son will change too. IMO that tone should have been set a long time ago. You are a single woman/single mom and that is not by choice yet you are accommodating your H way too much.
What have you put in place about your son spending time with the OW?
See, as it stands now your H has the best of both worlds. He has you to be an excellent and trustworthy full time caretaker of his son. He has OW for everything else. I would start making things a little less predictable with your H as far as your son goes. Perhaps then your H will get a good idea of how OW will be once she gets a full taste of what dating a single date with an exW is like.
While this is painful for me to type I have to say I agree that your H, at this time, has no intention of returning to you. Like many other WAS it seems he is doing all he can do to be easy and agreeable so you will eventually "get on board" with a divorce.
Think of all of this as a marketing issue. OW has marketed herself as fun, sexy, worthy of your H's love and affection and the best choice for a intimate and romantic partner. She has done this so effectively she has convinced YOUR husband that she is "the one" despite the fact it means he will get less time with his son AND his family does not approve of her. She has done a bang up job on her marketing campaign.
You have marketed yourself as a loyal, stable and wonderful mother. All very good things but not enough to keep a marriage in tact. Partner that with the fact you and your H do have emotional baggage, well, a good mom is just not enough.
I don't think there is much you can do at this point to really force your H to take pause as IMO you have been very meek/fearful to rock the boat too much and now a grand gesture may look manipulative.
What your H fell in love with (with you I mean) may not be attractive or appealing to him anymore. So you need to repackage yourself as a woman but not "for him" but for you. Looking nice is fine but you need more. What is your definition of the kind of WOMAN you want to be. I know you want to be a wife and mother and that is fine (and amazing and important) but what about the kind of WOMAN you want to be.
IMO you are afraid to really explore that part of yourself because it may not be the kind of W your H wants. So what? He left you while you were pregnant and is still fooling around with his mistress so clearly he is staunch in his choice. That is fine but don't make it so easy for him. You can still show him consequences while being polite.
