Bank can simply be raised earth or a hillside. 'The Banks of Sweet Primroses' is not a river bank, just a slope where grow lots of primroses. Could be a river bank but it doesn't have to be.

My money's on some hillsides in a park in Dundee, or even the banks of the sea there.

Catnach printed it as did many of the printers of the early 19thc but I haven't seen any pre-1800 versions.