Those are the correct lyrics. One person I knew thought I was singing "lone niggers linger by the door." I can understand how it could be hard to clearly hear that the words were "lonely girls" (the "-ly" syllable gets sort of swallowed) but I was shocked when she told me what she thought she heard. I guess she's not the only one, though.