Actually, I think Azizi's point about the use of "lady" was interesting. Even if some people consider it inappropriate, as it can be used as a put down, it obviously wasn't in this case, and that it would not have happened in the past in the States is informative. There's a huge difference between someone using "lady" when wanting to refer politely to a person whose name they don't know, and some smarmy blighter leering about "the ladies, god bless em", and most people would recognise that.
I'd much prefer it to being yelled at as "Miss" by ex-pupils who have forgotten my name. There are actually schools who teach that as a polite form of address (men being Sir) for their mostly married staff. That isn't as bad as mammying black women, but the intent was originally the same.
As for when Africans and Asians were here, I don't believe that they were visible after the Roman occupation, as there's no evidence for enclaves, and if they intermarried, the visibility would have diminished as has that of the descendants of the freed slaves of the 18th century. Definitely by Tudor times, as there is literary evidence for their presence, but they wouldn't have been obvious to most of the country.

Penny