I think it's worth distinguishing between labels with a built-in insult, and labels which are only seen as insults because they used as such - for example - "White Trash" or "Trailer Trash" on the one hand and "Poor White" on the other, along with "Black" and "Brown", which in themselves are no more or less insulting than "White".