I've had this problem with honey jars, when I've wanted to use them for making jam.

I've found White Spirit will generally do the trick. Just dampen a cloth with the White Spirit and rub the label. (If there's a shiny top layer that would stop the spirit getting through that needs to be peeled off, and that is normally easy enough to do.)

I think, in the States, White Spirit is called "Stoddard solvent". What a drab name. Useful stuff.