There were a fair number of black people around in England in the 18th century - slaves or servants or sailors. And then after the American War of Independence they were reinforced by exiled American veterans from the black regiments that fought against the rebellion, as a way of winning freedom from slavery.

The indications are that they mixed in with the general population, rather than maintaining a separate community. The chances are that by now a great many white English people have some black ancestors.

Interesting article here from the BBC Open University site - The First Black Britons.

One paragraph in that article to raise the spirit:

The black and white poor of this period were friends, not rivals. So much so, in fact, that Sir John Fielding, a magistrate, and brother of the novelist Henry Fielding, complained that when black domestic servants ran away and, as they often did, found ' ... the Mob on their side, it makes it not only difficult but dangerous to the Proprietor of these Slaves to recover the Possession of them, when once they are sported away'.