Well, you've always got to "do whatcha wanna." I hope you are not giving up any significant income that you might miss ~ that's the only good reason I can imagine for you perhaps having regrets after the fact.

The sooner you get back into the "spotlight" as a soloist, even at the humblest kind of open mike night, the sooner your chances of finding a new and better situation, either as a solo act in a paying venue or as part of a new and better-for-you group.