Hey, my boys are about to get their Bear badges too -
I was a den leader for 2 years; the Scouts are VERY accepting, remember, as long as you "do your best". Without really any visiblity into your situation, I'd encourage to keep on with the scouting. My $.02.