FWIW last Jan. when my H was at one of his meanest points he also told me he was thinking of "coming back" after the fact. Much later he admitted to me he was never thinking of coming back but his guilt was so overwhelming he told himself (and me) that to ease his mind.
I struggle with the same thing. My H has NO CLUE what it feels like to know your spouse is having an ongoing affair and the absolute pain it causes.
Maybe getting out of town for a few days will do you some good.