Often a genuine racist whose intent is to humiliate, and also to provoke fear in the person who is the butt of their joke, will use the blind loyalty of thrir cronies to hide behind.
Real racists in this respect are cowards. They tend not to have the bollocks to repeat their views when their bully boy mates aren't around to back them up.
The politics of racism is the politics of fear.
Those who are racist are afraid, whilst those they fear, they actively intimidate.
In this case, the protagonist had no guarantee of any support from anyone. There were many black people in the audience by the look of it and they had the freedom to react as they chose. In fact, the only person who may have been in any danger, (and I suspect he began to feel a little fear once it dawned on him what he had done) was Michael Richards.
Though even that last statement makes certain assumptions that might not bear the weight of careful scrutiny.