Oh, kat (sigh!) - poor Mr Grassfield and Mrs Worcester .... The past perfect is only "more correct" where it's more correct. The issue is not how far back in the past the action is in and of itself, but how far back it is in relation to some other action in the past. Typically, but not always of course, fiction is written as having happened in the past. Now, if the narrator, having established that he is in the past, then speaks of something further back in the past, prior to the linear main plotline as it unfolds, then out comes the past perfect tense (your "hads"). For example: "Before I tell you about how Aunt Agnes wrestled down a charging bull, you need to hear what little Lucy HAD said to her THE DAY BEFORE."

If, on the other hand, your narrator is speaking in the present tense, then he will use the simple past to speak of action that took place before the present action: "I can't believe [present tense] it - Aunt Agnes is in a wrestler's crouch, jaw clenched, awaiting the charging bull. I suddenly remember what little Lucy said [simple past] to her YESTERDAY."

However, you will notice that many writers will find ways around using the past perfect in situations where it would otherwise need to be used almost continually in a text, e.g., a relatively long passage relating a character's history prior to his entry into the story. When this is done, it is because the past perfect gets too "clunky" (assuming the writer is grammatically competent). You will notice that the past perfect is rare in modern poetry, for this reason.