(Here are the lyrics in the DT: https://mudcat.org/@displaysong.cfm?SongID=6580)

I've idly wondered where the name of this tune came from for a while. Though the authorship is fairly certain (words by English Nonconformist minister Philip Doddridge, first published after his death, in 1755; tune by Sarah Lancaster, of West Point, Georgia in 1869), most people seem murky about how it got its name; Fresno State notes that no source explains how the tune got its name but goes on to postulate that the words could be related to an unbeknownst preface added to his final work, De Revolutionibus, however this seems unlikely to me; you can read the preface here.

After a bit of searching, here's what I came up with as my attempt at an explanation:

It's generally agreed that Copernicus's last words were a paraphrasing of Luke 2:29 -- "Now, O Lord, set thy servant free." That seems to jive pretty well with the sentiment of the song.

Though not the same, verse somewhat similar to this song presented as "the last words of Copernicus" can be found in Memories of Father Healy of Little Bray (p. 185), supposedly translated from the titular Father James Healy, though I'm not sure what the source was given the disagreement with the common line above and the song in general. Nonetheless, I figured I'd include it:

Not the grace Thou gavest Paul,
Who saw Thy Stephen stoned;
Not the grace that Peter won
When blinding tears his crime aton’d;
But, ah, dear Saviour, give to me
The grace which Thou on Calvary
Didst give the thief who at Thy side
Repenting hung, repenting died."

Now, the one thing these all have in common is a certain acceptance of death and the futility of fighting it; this trend is also shared in the song itself. My assumption would be that Doddridge gave the song this title because it's a song about your soul leaving earth and ascending to Heaven, as well as the repeated planetary language. It seems a little simple, but given the song doesn't directly mention Copernicus himself at all, it seems the most likely answer to me. Would it be irreligious of me to cite Occam's razor? ;)

As a final aside, several texts of the song bear similar language in the subscripted verses; for example, this version fittingly relates the song to Revelations 22:5 - "They need no candle, neither light of the sun; for the Lord God giveth them light."