Your pastor is correct and I'm sure you know that.
My W took off the wedding band and then began wearing her grandmother's band. What did that mean? It meant what ever it meant, to her... maybe she liked the ring and it fit on that finger. maybe she was remembering her grandmother. maybe she didn't want to go out in the world as though she were single. maybe it was to show me that she felt unmarried to me but also unavailable to me... maybe she wore it to show her EA that she was unavailable to any other...
You are unlikely to get an answer that is satisfactory, if you were so bold enough to ask her.
Best you can do is ignore it... or at least assume it means nothing specifically, about you...