I read a lot of these as assignments in high school. I read "Moby Dick" on my own and it convinced me to head out to New England. So I went to New Bedford, Nantucket and also hit Providence. The following year I hit Salem after reading "The House of Seven Gables." That house actually exists, by the way.

I would add Mandelbrot's "The Fractal Geometry of Nature" and how could Newton's "Principia Mathematica" not be mentioned?