I imagine that the tendency to associate the word the word "black" with negative things would have come from the idea of light and darkness, with the latter being more threatening, since you can't see what is going on in the dark, and so forth. That would imply that the same kind of negative associations would be likely to occur in dark-skinned societies as in light-skinned ones, and I believe this is borne out in practice.

The use of "black" and "white" to refer to skin colours is really rather odd - it generally bears very little relationship to the actual skin colour of the people so termed.