As much as you might want to call BS on him, there is likely some validity in his claim.
There is a general rule of thumb that women hold emotional connection above physical connection whereas men hold physical connection above emotional connection. While he is likely still in an A, he might think it's OK, if it's not intercourse. Doesn't make it right, just could be what he thinks.
Add that your A was with his brother, and as mentioned earlier, it can really add some messy to the layers.
It's not my intention to put blame on you, just trying to help you understand why he might say that to you in a way that might actually be valid for him.
That said, he COULD be BSing you. If it's really such a deal breaker for him, then why has he not filed D?
No, don't bring these things up with him at all. Just note them as something he's said right now which he may or may not feel is truthful for him, right now.
Also, WAS / MLCers will sometimes say those types of things to push the LBS further away. It could be that he is feeling pressure from you, still.
Do you feel that could be the case?
Also, again try not to mind read, but there is certainly the possibility that misery loves company and if he's noticing you are happy, then he might have just said that to try to make you miserable like he is.
Again, let stuff like that go. If he's really so unhappy and your A was a deal breaker, he will file. Until the document is signed and filed, there is still hope.
