How many stories have we all read, or heard from our friends, of those that have been told that they can't sing but go on to prove their critics well and truly wrong?

It sounds as if your first teacher has done you a diservice, undermining your confidence, and later regretting it. Maybe she had an off day when she said those words.

This lack of confidence is because I know that I can't sing on key, no matter how much I tell myself "it doesn't matter, this about the words!".

Funny coincidence that - at a workshop I attended yesterday the presenter said the opposite. He had us "la-la-la-ing" to something -twinkle twinkle little star I think - and it sounded lovely. Then we started singing the words. The volume dropped and it was a little ragged. He said that the brain changes modes, suddenly thinking "I'm singing now, I have to make sure the words are right, have I got the tune right, am I in tune?"

Back in the early '70s there was a poem set to music which got a lot of commercial radio airplay. Desiderata is it's name. There's a bit of that which says:
"If you compare yourself to others, you may become vain and bitter; for always there will be greater and lesser persons than yourself."
It's much easier said than done to not make comparisons, and maybe it's not always the best way to go anyway but, if you are a relative beginner musically speaking, comparing yourself with professionals is not giving yourself a fair chance. Let their standard be something to aspire to by all means, but don't give yourself a hard time because you're not there yet.