if someone says immigrants of some particular ethnicity are all like this or that, they may be racist

But that's exactly what you're saying! You're saying that immigrants of any ethnicity other than English are all alien to English culture and a potential threat to it. You're a racist even by your own arbitrarily limited definition, I'm afraid.