More than once, stretching the hand to fit the frets on a banjo has been mentioned. I'm pretty I just read in a book (I read a lot) that it is bad to stretch a part of the body and then make it do work. Stretching to find a fret and then pressing on the string sounds like you're doing exactly that.

I have two thoughts:

If you have to stretch, make sure the strings are not too high off the fretboard. That could compensate some by reducing the amount of pressure required.

Perhaps you shouldn't be stretching towards the frets but releasing the hand and jumping to the new position. After all, banjos are noted for their pointillistic sound.

I've never played, I'm just wondering.