I vote for going too. You're giving them too much power. Your daughter just wants her parents at her graduation - and she has a good point, there WILL be other events in the future.
If you so desire, just text your ex and tell him to stay away from you at the graduation. But go - for your daughter's sake.
(BTW - just went to my son's graduation. Made an excuse so that my mom and I weren't available to join them for dinner after. Didn't sit with ex but did have to see him and his wife briefly before and after. His wife was not an affair partner so a little easier for me to take. I'm over the divorce, just mad at my ex for how he's treated our adult children. You can do this.)
