It's time to leave the band when they stop picking you up on your way to the gig. Which is what Pink Floyd did to Syd Barrett after David Gilmour joined.

Of course, Syd had been torturing them for months at rehearsals by making them learn a tune which he changed every time. He called it "Have You Got It Yet"...