Here's a start. The 18th century lexicographer Samuel Johmnson left £70 a year (a good income) to his servant Francis Barber. Black servants were not uncommon in 18th century England, some having been brought here as slaves (though seldom straight from Africa), and reference to them is not uncommon in pub names (The Black Boy, the Black's Head etc.)