A non-racist could have lots of reasons for questioning immigration/emigration, such as population density, land rights, trying to have a multiple number of cultures living under the one state law, caring for young Spanish couples who are being priced-out of the property market by wealthy English and Germans - not content to just visit as respectful tourists, etc.

A truly non-racist would see these things as in the light of the economic realities of 21st century capitalism which really have fuck all to do with immigration. Only a racist would attempt to use them as scapegoats to justify their anti-immigration cause.