> given the song's origins

But what are the origins that support that interpretation?

"Buffer" (fellow) appeared in the 18th century and was well known; but the suggested "Buffer" (member of the Buffs) seems not to be traceable anywhere.

If a specialized meaning of "buffer" is desired (though it's certainly not required) I'd suggest the naval "boatswain's mate," recorded in the 1870s.