I beg to differ with Ian C. I'm willing to bet that "willow" got changed to "yellow" by people who half-remembered the words. After all, willows are naturally sad, since they often weep, but yellow is not naturally sad. In fact, it's quite cheerful.