And you don't have to play chords that are major or minor anyway.

In many tunes you can use diads that don't include that third note - so instead of it being EAG for E minor, or EAG# for E major, you just play EA; and so forth.