What is the point of reading from a book is you are improvisiing anyway?

Because you're only improvising certain aspects of the song - expressive and melodic details, not the storyline or the bulk of the text.