Hey Don, I think you've got it!!!!! I have always heard the word as poverty (not ivory) the line makes sense, and that's the way I used to sing it until someone told me I had misheard it, and it was "with gates of ivory." I agree that the line as you hear it is not only right, and fits the feel of the whole song.

It doesn't hurt that that's the way I've always heard it, too...

Jerry