It would seem to me, that the cause of the skirmish at Charlottesville was indeed over the removal of the statues of Civil War "heros". Apparently the civil war was not fought simply to outlaw slavery, but for other political reasons.
Lincoln himself said that he was not in favour of the repeal and many of the founders of the country were massive slave owners.

All over Scotland we have monuments to people like William Wallace , Clan chiefs and generals who were considered terrorists by the UK government, but national heros by at least half of the Scottish people.
Can you imagine the uproar and civil violence which would be caused by the attempted removal of these monuments?