I don't think it takes much mental fortitude to understand how the song is about slavery; it was written by a slave, it was sung by slaves (and former slaves). While the song never says the word "slave," it's difficult for me to imagine that the subject wasn't prescient on the minds of every slave and ex-slave who sang it, hoping to find a freer life after death through salvation in Heaven.

That being said, I personally can't see a good reason why it can't also be sung at a rugby match. It's not glorifying slavery--it's hoping for its end. Unless there's some detail in the way they sing it, or their motivations for singing it, that I'm missing, to me it's not of much consequence.