I doubt that being billeted by the unit at a local inn still would put him "along side of" it in any natural use of language.

No text says, "I live at the sign - and, by the way, along side - of the Ups and Downs." It's one or the other.

(He might have said, "I live along o' the Ups and Downs." That, of course, would be different. But he doesn't.)

And if the inn has a false name, it isn't real, and he doesn't have to be a soldier of the "Ups and Downs" or any other regiment. Conceivably he's just a bloke who lives nearby.

James Reeves (of "Foggy, Foggy Dew" fame) seems to have connected the song to the "69th" back in 1958.

Because it's fun to believe (adds corroborating detail, wot?), it seems never to have been challenged.