Wikipedia says that this song was not originally meant to be sung at Christmas, but was dance music for French nobility. The carol was first published in France in 1553, and came into English in the 18th c.
The song tells the story of two milkmaids who went to milk their cows in a manger in Bethlehem, only to discover the baby Jesus sleeping in the hay. The two girls ran to the village and the townspeople came with torches to see for themselves.
"To this day, in the Provence region, children dress up as shepherds and milkmaids, carrying torches to midnight mass on Christmas Eve, singing the carol."

I have not found a French site with the story.