That 'love you, but not in love with you' business is the WAS comparing apples and oranges. They are in Stage 1 of their new relationship, vis a vis the affair, and in that infatuation stage, fireworks go off, everything about the OP is enhanced, their hearts thump when they see each other, they think about the other all the time, they put blinders on as to the OP's faults, etc. They don't feel that way about their relationship with you, because your relationship is past that stage. It's not a fair comparison. But infatuation doesn't last forever. It can't for several reasons. Anyway, their mind reasons that if they feel so gaga over the OP, but not you, then they reason they must be in love with the OP, and therefore, if they're in love with someone else, then they can't possibly be in love with you, though they still care for you.
None of this is really true, it's how they feel at the moment. Consider that relationships begin many times with such a heightened state of feelings where the OP can do no wrong and this is it! this time it's for real! (even though this is exactly how they felt at the start of their last relationships) and they will now be in love forever, but that in time, many such relationships end with one or both of the parties declaring that the OP is such an a$$hole and the relationship falls apart, right?