People vary. Stop shaving and see what happens. If it doesn't grow quickly, and having a fluffy chin instead of a hairy chin worries you, accept it you'll have to wait a few more years.

Men don't actually grow beards. By which I mean, it's not an action in itself. A beard is what happens when a man stops shaving the lower end of his head - or not, as the case may be.