"can't you just follow the path the message has taken in the Received lines of the header"

You could, but it's not hard to fake some of them too...