I went looking for lyrics on the 'Net and found that someone has written a book called When Ladies Go A-Thieving: Middle-Class Shoplifters in the Victorian Department Store. There are lots of references to the book, but no lyrics that I can find. I wonder if the book contains the lyrics?

By the way, AMG says the song by the Bandoggs is called "Ladies Don't Go Thieving." It is also shown that way on the playlists of a few radio stations. (Or else "Ladies, Don't Go Thieving." That comma makes a big difference!)