I always understood 'black in the face' to mean angry.
The line I know is:
"she may scold and may grumble, 'til she's black as coal"
To me that fits fine with the vernacular. About as racist as 'giving someone black looks'.
It is possible to be oversensitive.