Objection to immigration on cultural grounds may or may not be racist, depending on to what extent and in what specific regards you are concerned about cultural issues.

Every population in the world has become annoyed or fearful at one time or another over various troubling cultural issues that arose out of immigration. So what? That happens. It happens everywhere. It happens both ways (in the breasts of both locals and immigrants). It doesn't necessarily indicate that people are "racists", it indicates that they are uncomfortable with customs and behaviours that are unfamiliar to them.

For instance, in Canada the following issue has arisen:

Should Sikh police officers in Canada be allowed to wear a turban instead of the standard police headgear while on duty? (I say "yes, sure, if they want to". Many people say "No, because if they want that job they should be willing to wear the normal uniform.")

That's not a "racist" issue, no matter how you stand on it. It's a cultural issue of what people think is right and proper normal behaviour while on the job. To a traditional Sikh, it's improper not to wear his turban. To the average traditional Canadian non-Sikh, it's improper not to wear the normal police headgear.

Neither one is being racist, but they are both clinging to cultural habits that mean something to them.

I simply don't give a damn one way or the other, so my inclination would be to let Sikhs wear turbans while on duty if they want to. I don't see why it matters...but I am more flexible on traditions than most people are.

You see, it's not that I'm necessarily less racist than they are...it's that I cling less rigidly to established tradition, that's all. To cling to traditions and rules and to expect others to do that also does not equate to being a racist...it simply equates to being somewhat inflexible in an emotional and behavioural sense...and most people are like that.