Although I was taught as a child, I too am functionally illiterate when it comes to written music. Which has never appeared a significant handicap to me. Malcolm has it right, I think: We learn to speak by hearing, and learning to read is a useful thing that comes later. The same is, or should be, true of music.

That implies that those who teach music theory first, and listening second (or not at all), have it backwards. An understanding of music is best acquired by listening -- as much and as variedly as possible.

LilyFestre -- you say you play fiddle, so I will boldly surmise that you want to learn traditional fiddle tunes. You can of course learn the basic skeleton of any tune by reading the sheet music. But what brings the tune to life are the things that can't be notated: the swing, the lift, the neagh (Irish for "that certain something"). You assimilate that sort of thing by osmosis, by sitting in sessions and listening to records. It can't be taught. The dots should be only reminders of "how does this one start?"

CarolC is right as well, of course, saying that our brains are all wired differently. That's why to some it comes easy, and some never get it at all...

Cheers,
Michael