Your problem, and it is your problem, is that you "know" blackface is related to minstrelsy. I doubt, but do not claim to know, that any survey done would find that almost none of the population have any clue as to what minstrelsy was.

If I take offence at something I either, depending on the situation, tell those as much, or simply walk away.