Joerg, the reason a flatpick is traditionally used is to get the "attack" that only a flatpick can provide. It's very difficult to get it to sound even when fingerpicking because the fingers and thumb have different attacks. Still, you can find examples of bluegrass guitar played with a thumbpick and two finger picks. Listen to some of the old Flatt and Scruggs recordings. I'm specifically thinking of a tune called "Preachin', Prayin', Singin'" where Earl takes short guitar breaks. There are other examples of his playing this way, I just can't think of any right now.
As for the banjo, there are more knowledgeable folks here than me but my understanding is that the fifth string was always part of the banjo until the advent of the jazz age spawned the creation of the tenor or four stringed banjo. The Gibson Company always designated their five stringed models "RB" for regular banjo to differentiate them from the tenor banjos which were a foreign instrument in the hands of a five-string player. The fifth string (or thumb string) only reaches part way up the neck because it wasn't desinged to be noted. It's just a drone string serving a purpose similar to the drone pipes of a bagpipe or the drone strings of a mountain dulcimer or a hardanger (sp?) fiddle. It's the fifth string that gives the banjo it's characteristic feel and cadence. In Scruggs style playing, the fifth string is normally integrated into the arpeggio of the roll and doesn't often play a melody note. In the chromatic styles of Bobby Thompson and Bill Keith, it is used as a melody note just as any other string. If you're unfamiliar with the five-stringed banjo, you should know that it is typically only played in the single key to which it has been tuned. If you want to play in a different key, you must change your tuning.
Hope this helps,