"Mary, Don't You Weep" is a song of rejoicing that evil is punished.
Lyrics are here: @displaysong.cfm?SongID=4365

I've noticed this theme in songs recorded in the Bible, including three sung by or composed by women:

"The Song of Miriam" regarding the punishment of Egypt:
Exodus 15: 21b
      "Sing to the LORD,
       for he is highly exalted.
       The horse and its rider
       he has hurled into the sea."

"The Song of Deborah" regarding the punishment of Sisera:
Judges 5:1-31 (24-31a below)
      "Most blessed of women be Jael,
       the wife of Heber the Kenite,
       most blessed of tent-dwelling women.
       He asked for water, and she gave him milk;
       in a bowl fit for nobles she brought him curdled milk.
       Her hand reached for the tent peg,
       her right hand for the workman's hammer.
       She struck Sisera, she crushed his head,
       she shattered and pierced his temple.
       At her feet he sank,
       he fell; there he lay.
       At her feet he sank, he fell;
       where he sank, there he fell-dead.
       Through the window peered Sisera's mother;
       behind the lattice she cried out,
       'Why is his chariot so long in coming?
       Why is the clatter of his chariots delayed?'
       The wisest of her ladies answer her;
       indeed, she keeps saying to herself,
       'Are they not finding and dividing the spoils:
       a girl or two for each man,
       colorful garments as plunder for Sisera,
       colorful garments embroidered,
       highly embroidered garments for my neck—
       all this as plunder?'
       So may all your enemies perish, O LORD!
       But may they who love you be like the sun
       when it rises in its strength."

"The Magnificat" regarding the blessings to be brought by Jesus:
Luke 1:46-55 (Luke 1:51,52 below)
      "He has performed mighty deeds with his arm;
      he has scattered those who are proud in their inmost thoughts.
      He has brought down rulers from their thrones
      but has lifted up the humble."

What other examples of songs that rejoice in the punishment of evil come to mind?

Kent