Close, anyway...the lines go like this:

When the night has been too lonely,
And the road has been too long
And you think that love is only
For the lucky and the strong

But I confess, I have always heard it in my mind just the way you described it.