It seems like he would rather completely forgo the time rather than make a compromise. It's like he has a pathological inability to 'lose'.
Pathological is the operative word here. Also, as I predicted, once you offered him more time, he would decline to take it because, whoa, that would mean he has to do the WORK of parenting.
And honestly, although I know you are trying to foster a good relationship between your son and his father - the truth is, some dads are better off as minimal "Disney" dads. Seriously - what of any value is he going to learn from this guy as his father? I'd just document all the time that he has declined custody, and the fact that he doesn't want to take any weekends, for the courts. And unless S2 is begging to see daddy, just let it become the norm that "daddy" is just a sometime thing. Find some better male role models for S2 in his life as he grows up. (One young woman I know is raising her two sons pretty much by herself without much help from the two deadbest who fathered them, but her dad has stepped in as a great role model. My niece, as I have recounted before, had minimal visitation with her uninvolved biodad growing up but a great relationship with the stepfather who married my sister when the niece was 6. )