I play in a Bluegrass band, and our 'concert' music is often very fast indeed. We also play for barn dances and other dance events; the tempo is very different, and we don't have a drummer. I think the key is to imagine yourself dancing as you are playing, and to watch the dancers as you play. It's also useful to have a system of communication between band members, such as gestures or other movements that suggest 'faster' or 'slower'. The rhythm in our band is generally set by the bass player (who does the 1 and 3 beats) and me on mandolin (doing chops on 2 and 4). It's also useful to be able to play a tune at a 'very slow' tempo, in case the caller wants to walk the dancers through a particularly difficult set of moves.
As always, a lot of it comes down to practice and experience, and it does get easier the more you do it.