Words are difficult in this situation because it's a non-verbal communication. Words would only call attention to your discomfort, which - in some cases - you don't want to do.
Invading personal space occurs for a lot of reasons; sometimes it's indicative of too little space (a crowded room), someone trying to assert dominance over you, someone trying to gain intimacy, an indication of a person's need (imagine yourself asking someone for a loan you desperately need), or just a real lack of interpersonal skills. Whatever the case, it's hard to deal with. A lot depends on your tolerance for discomfort in asserting yourself.
If it's not a crowded space, the first step is obviously to step back away from the person. If that doesn't work (the person closes the space), you can suggest you carry on the conversation in a less crowded part of the room, and walk away to such a place. If that doesn't work, I'd say politeness can be discarded.
One maneuver you can try is gently putting your hand on the person's arm (especially useful if the arms are crossed) or if they are very close, their shoulder, and backing away again, keeping your hand on the offender. This will cause your arm to extend a bit. If they try to close the space, keep your arm extended. That is a loud and clear "keep your distance" in body language. (President Trump likes to invade people's personal space by pulling them close to him. It obviously makes some people very uncomfortable. Prime Minister Justin Trudeau used this maneuver to avoid his being pulled by Trump.)
The only way a person can close the space while you're doing this is to shake you off. If that happened to me, I'd just say, "Excuse me, there's someone over there I need to talk to," and get the heck out of Dodge. I really wouldn't worry about offending them, because I would feel unsafe with that person and I'd want to keep clear of them anyway.
If that feels too assertive to you, I think politely excusing yourself as soon as possible is your best option. If they are trying to assert dominance over you, and you admit you're uncomfortable, they've scored a touchdown.
If you like the person and you're worried about them (are they mentally ill?), please know that mental illness makes a person increase personal space, as does anxiety. But if this person does it to everybody, no matter what they're talking about, then I think they may just be socially awkward and you can gently tell them that you'd appreciate a little more space when the two of you talk. If it doesn't happen, you've done what you could.