What if the next time he did that while you were talking, you text him or email him if that's what he's checking out on his phone finishing up what you were saying? In a "If you can't beat 'em, join 'em" kind of way. And when he realizes it's from you, just give him a grin and busy yourself watching TV or doing something else.
The guy really seems to be lacking in basic manners and consideration, which given what you've shared about his father, was probably ingrained in him long ago. Doesn't mean he isn't capable of learning some basic socialization skills, but remember it isn't because he's bored to tears by you. It's some combo of self centeredness and lack of learning how to interact with people in a respectful way, but it's not about you--remember that, ok? I can manage to read an entire post from you without being distracted by the cobweb in the corner or the cat crawling on me or even, gasp, my phone vibrating next to me because there's a message and I've never been married to you! LOL
Me38,H:38,S:7 Married:6/99 Bomb:7/04 Sep.:5/05 D Filed:3/08;Final 1/10 Piecing:11/09 H moved back:09/10 Current thread: http://tiny.cc/htcty