I read somewhere that chimney sweeps had a fertility connotation at one time. (I presume for the blatantly obvious reason.) This would be in England. Don't know if it applied anywhere else.

The Irish would sometimes describe someone as a "black (insert word or insult)". I don't think this has anything to do with race though. The most common use (and it'd be dying out) would be "black protestant" or "black presbeterian". I imagine there'd be a link to blackguard as well. Might support the whole devil thing.