What is it like to be repeatedly asked where you are from?

It is a question that ethnic minorities are constantly asked, but why exactly is it so loaded?