This is not a freedom of speech issue.

A racist comment made in any workplace is grounds for reprimand at least if not for dismissal. If such a comment makes any colleague uncomfortable they have a right to seek redress, whether or not it is aimed at them specifically.

Check the Equaltiy and Diversity policy where you work.