To be honest, as long as you are happy with your system as it is, I see no reason why you shouldn't delete the folders that are something like "$uninstall". The only problems I can think of is 1, you wouldn't then be able to revert to the previous setup (I don't see that as a problem as long as you have convinced yourself the hotfix hasn't mucked something else up on your system) and 2, I think you would be stuck with a registry entry that Control Panel uses for the add/remove programs entry.

I've a load I'd like to get rid of. I'll try and do a bit more research and report back. I suspect somewhere like annoyances.org will probably have a solution.