You're right about the Peter Pan, McGrath, but I think it's the I can't...but that's the really tricky part, because it defines the speaker by what s/he can't do, first.
But if the song is about flights of fancy in general, and the ability of the spirit to soar no matter what frustrations we face, then we're entering the realm of the universal...
But, BTW, that is another pet peeve of mine: that flights of fancy and believe in magic are allowed only to children: that Never-Never Land can only be inhabited by lost "boys", or that Puff the Magic Dragon must face a lonely, solitary eternity when Jackie Paper grows up, or that Christopher Robin must abandon the Hundred Acre Wood when he goes to school.
If the magic wand is a metaphor for personal power, and Never-Never land is the realm of possibility, then shouldn't grown-ups take those things with them into their everyday world, rather than leave them behind?
Perhaps that is one reason why the Harry Potter books are so popular, at least on one subconscious level: in that world, childhood is the training ground for magic, and even though we won't see any more of Harry after he graduates, we know that when he grows up, he'll be taking the magic with him.
Actually, there is already a fantastic song on this topic: Eleven Easy Pieces by Keith Grimwood and Ezra Idlet (aka Trout Fishing in America) -- a song that proves good lyrics don't have to rhyme, even though it may not come across as wonderfully on the screen as it does in the ear.