"Logically" it could have been a US flag, or a Soviet Flag, a UN Flag, any flag you cared to name. Just because something is logically possible doesn't make it reasonable to think it's true.

But there is something flag-like about the song - like a flag it is pretty unspecific, it takes its meaning from the way its used, and from the meaning the people using it give to it.

That's not a criticism, its how certain types of songs are, and its part of their strength. In principle there's nothing in a song like "We shall not be moved" which would get in the way of it being used by hardline segregationists, with minimal adjustments to the words customarily sung.