The companion song from the South, "We'll hang Abe Lincoln...."

Lots of verses, all to the air- "Brother, will you meet me? (Glory, Glory, Hallelujah")
Here is one from American Memory:

Ellsworth's Body Lies Mouldering in the Grave

Ellsworth's body lies mouldering in the dust, (3x)
As we go marching on- Glory, Glory, Halelujah!

He's gone to be a soldier in the army of the Lord, (3x)
His soul is marching on- Glory, etc.

Ellsworth's knapsack is strapped upon his back, (3x)
As we go marching on- Glory, etc.

We'll hang Jeff Davis on a sour apple tree, (3x)
As we go marching on- Glory, etc.

His pet lambs will meet him on the way, (3x)
As we go marching on- Glory, etc.

The Fire-Zouves are marching on their way (3x)
For Ellsworth's death to avenge- Glory, etc.

Now three rousing cheers for the Union (3x)
As we go marching on- Glory, etc.

Song sheet, A. W. Auner, Philadelphia.