There's a number of secular songs that seem to be adaptations of the religious song "Down By The Riverside". This Mudcat thread has some interesting discussion of this song: RE: Origins: history of Down by the Riverside

The line "aint gonna grieve, my Lord, no more" [my punctuation] is found in a number of children's rhymes including the "Oh you'll never get to heaven" taunts such as:

Oh you'll never get to heaven
with ____'s face
cause ____'s face
is ah disgrace.
Oh you'll never get to heaven with ___ face
'cause ___'s face is a disgrace
Ain't gonna grieve, my Lord, no more.

-snip-

Around 2001 or so, I learned a rhyme called "The Preacher Went Down To The Cellar" from a young African American woman in Pittsburgh, PA area who said that she learned this rhyme when she was a child. This rhyme also has the line "Ain't gonna grieve ,my Lord, no more":

It goes like this:

Oh the preacher went down
to the cellar to pray.
But he got drunk
and stayed all day.
Oh the preacher went down to the cellar to pray
but he got drunk and stayed all day.
Ain't gonna grieve, my Lord, no more.