I am somebody who reacts as stated in the question; based on personal experience, I can answer the question as follows.
I very much dislike being asked where I'm from, because I don't know how to answer it, and any answer would give misleading information as I have lived in many different countries and am influenced to differing extents by all of them.
So I think yes- it's a bad question. Perhaps in particular when people tour India seriously, they might want to feel a part of the place, and the question, "Where are you from?" implicitly asserts that they are outsiders.
You could instead ask, for example, in what countries (plural) they have lived.