Perjoratives are common to every language. Pagan meant rural, rustic, but, as Stilly says, in Christian Latin it acquired the meaning of someone who preserved the old idolatry.
In Navajo, anaasazi, anglicized to Anasazi, meant an alien, an enemy, and it was applied to the people who had long before abandoned their settlements in what is now Navajo country. The archaeologists took up the word, and now the ancestral pueblo Indians (who these old peoples were) are known as Anasazi everywhere.

Whether the May-day 'survivals' have anything to do with pre-Christian festivals, or have evolved from medieval celebrations and have had 'pagan' attributes added to them by lovers of old fables is a fertile field for argument.