Travel isn't a fundamentally problematic topic. Still, even assuming this somehow fits into conversation and isn't entirely out of the blue, the answer depends on context.
Not everyone has the privilege of being able to afford intercontinental travel, so asking about travel may have the effect of highlighting a wealth/income disparity between you and the other person. In particular, framing the question casually ("have you ever been to Paris?") suggests that you think that it's not particularly unusual - and thus that you have the money to not be too concerned about that scale of expense, which may be very different from the other person's experience.
Money is certainly a sensitive topic in the US, so in some situations, that kind of question could certainly embarrass someone, or perhaps more accurately, be reasonably seen as inconsiderate or even offensive. I'd suggest being mindful about this kind of question, just as you might be mindful about discussing other expensive activities or purchases.
There are also very rational reasons specific individuals might be reluctant to travel to certain locations. Paris and New York are relatively unlikely cities for this to be the case, but for example, there are at the very least dozens of countries where it would be dangerous for me to hold hands with my wife in public. I would similarly suggest mindfulness in that regard.
But asking whether someone's been to a nearby town, or asking about a particular destination in a conversation that's already about travel on that scale, with everyone participating? That's fine.