A boundary IS an ultimatum.What comes at the other side of crossing that line as a consequence makes it an ultimatum. No matter how the words are disguised.
On the board there is so much talking about what our spouses do that we often forget to talk about what we did that got us here and what we may or may not still be doing.
Your wife did not intend to ever come back. And like Jack said, you screwed with it. It appears she still does not intend to come back, although she is entertaining the idea from time to time. She is also entertaining the idea of not coming back and deciding who she is going to deepen a relationship with, if anyone.
You are most likely NOT in a position to set ultimatums/boundaries and I think you know it. The other guy is NOT your biggest problem. And you know that too. A love that is compelling enough for her to come home to is your goal. Whatever speaks love to her most.
You may feel you are being crucified for your past, but you are not. Your relationship dynamics drove her away. You are not still paying for it, you have done a great job of drawing her back in all over again, you just haven't drawn her 'ALL IN'. The good news is, the OM hasn't yet either.