Frank,Welcome into the fraternity. As mentioned above, your best defence is offence. Learn as many banjo jokes as you can so you can head your detractors off at the pass.
They can be extremely frustrating, but stick with it. Find a good teacher, listen to lots of Ralph Stanley and Earl Scruggs.
Check this site out, it might be helpful.