According to my old English Lit book, Byron was a dissolute and promiscuous aristocrat who was nevertheless adored by many. He had many affairs with married women, and a long-term incestuous relationship with his half-sister who was raised apart from him. One section from the original poem which was modified in the song to significantly change the meaning:

Thy vows are all broken, and light is thy fame
I hear thy name spoken and share in its shame.