I think the relief can mean a lot of things. Sometimes the person who's died may have caused stress, anger, other bad stuff. Sometimes there's a lot of pain in their lives, as well as yours, and their death really is a relief.
I guess this is as good a place to post this as any. Maybe there's some guilt on the part of those of us left behind, and we're trying to make up for it. Maybe you just wish you could have/would have done something when there alive, and you can't, so you try to make it right after they're dead. I recently went looking for someone I'd corresponded with a few times a couple of years ago. I hadn't seen any messages (newsgroup, not here) from him. I found one small entry on a web page saying he died in 1999.
There were none of the songs he'd written, no eulogy, no other mention of him. I didn't always agree with him, and I doubt we would have been good friends if we'd met. I did disagree with him politely, unlike a few others, and he did the same with me. I never disliked him, but I don't like him any more now that he's dead. I am left feeling sorry that some people treated him badly, even if that happens to everybody. I'll admit his imperfections, but I'd rather remember his good qualities.
So maybe some of it is us wishing we could right past wrongs, and maybe some of it is simply wanting to remember good things, because it makes us feel better.