How do I politely deal with this?

You can be polite yet assertive and say "I don't have time to attend the party because I have to study for the exam" and like you said, just go say goodbye to the person on your own time. Most likely, they won't care whether you do or not. You don't need to explain any more to them than what I said. They aren't going to interrogate you as to why you won't be attending. It isn't mandatory.