I've always assumed that the 'you' is a long gone or maybe near at hand beloved. We know who 'The first time ever' was written for; not known in this song - maybe it was just a song?