I too drift up, been doing it for years almost a comfort to see it discussed like this
What is nice is the number of tolerant listeners out there as proved by many of the replies to this thread
I would say that this tolerance is widespread because I generally got a sing when I turned up at a club despite this tendency to drift. So there is a lot of sense in the above posts saying get on with it and stop worrying.

On the down side this drift tendency used to drive me wild
It still gives me the pip when I get a song and it seems to naturally send the pitch up
I now analyse the song to find the section where the pitch changes and conciously try to sort it out. Sometimes I just lack the vocal skill to do that and as a result the song just gets dumped.

The good news is that I still drift but its far less than in the past.

Somewhere the problem diminished.