When people talk about 'mainstream' in terms of music it's about identifying what is most widespread, not about denigrating music that only has a minority appeal.

It would be absurd in this society to talk about folk music as 'mainstream' though perhaps I would wish it to be so. I might object to someone saying that my taste in music is abnormal (though I couldn't care less if they did, so long as they didn't interfere with my freedom to play it and hear it) but 'not mainstream', that's a different thing.