Like Charles Taylor, I had thought of "Morning Has Broken". However, unfortunately for the Atheist Hymnal concept, the lyrics include the phrase "God's recreation of the new day" and references to "His feet", and to creation theory (alluding to "Eden" and "the first morning"). There's also a hint of that annoying God-created-the-universe-just-for-me attitude in the phrase "mine is the sunlight, mine is the morning".

It's a beautiful song, but it ain't atheist!