Ah, but hold on, "meself": is it really justified to imagine them habitually "leaning", at least from the words? It's the male/singer's shoulder that is "leaning", which I take to imply a certain dejection at the time (and indeed, I've heard the word sung as "drooping" and "weary", though Yeats' word is "leaning", going along with the way she "laid" her hand &c). I'm thoroughly in accord with your third sentence, not least in the number and variety of possible explanations, but do tend to see the singer as remembering youthful experience from a long time ago, which does lead to the complication of wondering why he's (still) full of tears, presumably about the experience mentioned.

On the other hand, it's a song that works without any need for such analysis.