Interesting question, Marion. I would follow what the lead player was doing (or try) figuring that I was supposed to be following them. If it were a rehearsal session, I'd ask if they were intentionally changing tempo. In a performance I think trying to stay on the rythym could sound jarring to the audience and possibly make the lead player look bad.