It's based on a story in The Gospel of Pseudo-Matthew, where it was a palm tree:

Chapter 20.

And it came to pass on the third day of their journey, while they were walking, that the blessed Mary was fatigued by the excessive heat of the sun in the desert; and seeing a palm tree, she said to Joseph: Let me rest a little under the shade of this tree. Joseph therefore made haste, and led her to the palm, and made her come down from her beast. And as the blessed Mary was sitting there, she looked up to the foliage of the palm, and saw it full of fruit, and said to Joseph: I wish it were possible to get some of the fruit of this palm. And Joseph said to her: I wonder that thou sayest this, when thou seest how high the palm tree is; and that thou thinkest of eating of its fruit. I am thinking more of the want of water, because the skins are now empty, and we have none wherewith to refresh ourselves and our cattle. Then the child Jesus, with a joyful countenance, reposing in the bosom of His mother, said to the palm: O tree, bend thy branches, and refresh my mother with thy fruit. And immediately at these words the palm bent its top down to the very feet of the blessed Mary; and they gathered from it fruit, with which they were all refreshed. And after they had gathered all its fruit, it remained bent down, waiting the order to rise from Him who bad commanded it to stoop. Then Jesus said to it: Raise thyself, O palm tree, and be strong, and be the companion of my trees, which are in the paradise of my Father; and open from thy roots a vein of water which has been hid in the earth, and let the waters flow, so that we may be satisfied from thee. And it rose up immediately, and at its root there began to come forth a spring of water exceedingly clear and cool and sparkling. And when they saw the spring of water, they rejoiced with great joy, and were satisfied, themselves and all their cattle and their beasts. Wherefore they gave thanks to God.

~Masato