...it would appear that the remarks were only "racist" and "offensive" to those who sought to gain political capital by portraying them as being "racist" and "offensive".

Which is precisely what has happened apropos of "antisemitism" in the Labour Party, in that case the whole thing blown out of proportion by right-wing and pro-Israel factions both inside and outside the Labour Party in order to gain political capital.