Maybe because of weeping willow, willows often seem to be associated with sadness, as Pete says--for example, in "All Around my Hat" and Desdemona's song in Othello.