You got it exactly. When NO ONE SPEAKS up against her she just hears everyone telling her to keep going. The problem is people think that their silence says nothing :
SILENCE IS THE LOUDEST NOISE in the WORLD
Their silence is sending the the green light, and they need to hear that point.
In particular the pastor is misbehaving by remaining silent, everyone looks to him for guidance, and if the only thing he is offering them is silenc,e that's all anyone else is going to offer.
