Maybe you could have "other plans" that "just came up" for Sunday if you don't feel like going. The last thing you want is to convey that you don't want to be there, which you run the risk of doing if you genuinely don't want to be there. Meet up with her when it works for you.
