OK, here is my oversimplification, as a Charlottesville resident for decades:
It was the people who wanted the statues to remain marching through my town with torches in an attempt to terrorize us into not removing them that made me want to remove them.

And that was BEFORE the KKK came and reinforced my desire not to give them anything they want.

That was the month before the they came back and marched through my university with torches in another attempt to terrorize us into agreeing with them.

And after THAT, they committed murder, in an attempt to terrorize us into agreeing with them.

They want me and half my family, dead.
They want my African and black friends and family, dead.
They want my friends and family with accents in English, dead.
They want my friends' newborn baby, dead.

What more do they have to do for you not want them to get what they want?