The other reason he might have been hiding is that, let's face it: It's one thing to hide among a crowd of people, where Ken is not likely to make a scene, like at church. It's a different story to actually have to deal with someone one on one!