Both punctuations and a third are found in print and sheet music; 19th c or earlier origins.
Two versions in "The Oxford Book of Carols;" version 11 "God rest you merry, Gentlemen," version 12, "God rest you merry gentlemen" with no punctuation; and "God rest you, merry gentlemen" in the earliest, Roxburghe Coll. and reprinted in Chappell, 1859, "Popular Music of the Olden Time."

God rest you, comma, has age on its side.

That "ye" business is modern choral practice but none of the old versions used it.