You're obsessing. Thinking about the OM is going to drive you crazy.
And asking her out, then bringing it up again, then again is pursuit and pressure. She sounds like she's dragging her feet, that she isn't real excited about dating you but doesn't want to hurt your feelings. You've asked her, now leave it alone until she brings it up, if she ever does.
Know what I see? Your wife with the choice between a married man that is a challenge and her H/XH that is chasing her around like crazy. You are just too obtainable. I suggest you get distance and get less obtainable without being unfriendly. Quit asking her out and start working more on yourself.
In the end, it's not the years in your life that count. It's the life in your years. Abraham Lincoln
It is hard to fail, but it is worse never to have tried to succeed. Theodore Roosevelt