Hi, Arnie - we have Sail Away Ladies, which ends with "You'll be an angel by and by. That one has been recorded by Uncle Dave Macon, Joan Baez, the Rooftop Singers, the Kingston Trio, and others.
That's not the only possibility. This page (click) speaks of a song called "You'll Be an Angel By and By," recorded by Nellie Richards - I couldn't find out anything else about that one. Do you remember any other snippets of the song?
-Joe Offer-