I'd like to know just what sort of commitment has your H made to your child before and after birth that he thinks he has any real input or say into his upbringing. Looks to me like he's been an absent father, not just an absent H, so beyond a minimal level of supervised visitation, what on Earth makes him think he is due an entire day with a child who is too young to be separated from its mother?
Before anyone says anything, I am a father of two and a staunch advocate for shared parenting. But given that it appears from your story that your S's father had already made his mental and spiritual exit from the family prior to his birth -- probably even before his conception -- I don't think he has any believable claims to want more than the absolute minimum and age-appropriate visitation with this baby. And as the child matures, I think the onus is upon your H to prove he's anything more than a (dare I say it) sperm donor. Actions speak louder than words.