He probably was, Paul.
Imagination is one source of fear, yes, but it doesn't account for the huge differences in the amount of emotion that can be invoked by imagining different things.
There are also other things at play in the responses, Horatio.