'But "downloaders" are just listening and sharing.'

Yes, they are. And those they are sharing with, including themselves, have scoffed someone's work for free.