I learned how to read music as a 5-year-old on the piano. Always felt like those people who could play without sheet music were the REAL musicians. A few decades later, I learned to play guitar and fiddle by ear. It's much more fun this way, and easier (for me). People who do one are typically intimidated by the other. I took 1 year of classical violin lessons, and my teacher (who plays with the New Haven Symphony) was amazed at my ability to play by ear. She was completely unnerved at a jam I dragged her to, when called on to take a break. And I was amazed at her ability to take a piece of music almost completely black with notes and swim through it with feeling and interpretation.Being able to read music is a tremendous advantage. I can buy songbooks and find wonderful tunes in them that I would never know otherwise. I can pick up classical music and thoroughly enjoy myself on the fiddle/violin. I can read how the great composers intended their pieces to be played. And when I write a song, I can put the music onto my MusicTime software and print it down.
Being able to play by ear is a tremendous advantage. I can walk up to a group of strangers playing something I've never heard before and join in a jam. I can play in the dark. I can create music more spontaneously with minimum setup.
Anyway you can do it is fine, and any limits you set on yourself are your own. I recommend teaching children (and adults!) to do both.