I really don't think that it is the music that has the problem. It the idea that if you like the music you have to be a Folkie. The image of the Folkie has been pretty well described in this thread and is it any wonder that normal (whatever that is) people are a little wary of being thought of as a Folkie?
The children of Folkies do seem to adopt a different image, even if they continue to play and like the music? So maybe there is yet hope?