Well, I'll just chime in with the previous posters. It won't work with the boyfriend still in the picture. Bf may not see it quite that way...
And you are likely to be left feeling--I don't know what. To put it bluntly, one of the possibilities I would feel anxiety about, in your situation, would be dismissing the bf, only to be unable to make any real progress with Ex-husband. Essentially what you will be doing in meeting your ex's request--which certainly seems reasonable enough--will be one big leap of faith for you. To work on your former relationship with your ex, with only a very bitter, dismissed boyfriend as backup should you fail, does amount to a significant offering of your trust.
Somebody has to start somewhere with the trust. I think it's up to you. With the boyfriend definitely gone, you and your ex might actually make extremely good progress.
You haven't seen it from the other side. Although your ex's relationship with the house guest was inappropriate, and may have been wending its way toward an emotional attachment with boundaries torn down--it's still the physical affairs that seem to be the most devastating to the other spouse.
I'm really hoping this works out for you. You need an excellent counselor, I think. Someone with experience in marriage counseling--lots of experience--who is committed to helping you put the relationship back together.