It is my understanding that the phrase "in storms the riders rattle" refers to the method of construction of corrugated metal roofs that were used in bush homesteads at the time. Riders was the term given to the battens that were nailed horizontally across the rafters to which the sheets of corrugated iron were then nailed. The fasteners used nowadays are designed to hold securely to the riders, but back then ordinary galvanised nails were used and the wind would over time cause the nails to lift and required regular tightening down with a hammer to ensure that in storms the metal sheets of the roof didn't rattle and vibrate and create an infernal din.
To reduce confusion and to make this beautiful song more accessible, when I sing the song I change the phrase to "in storms the roof it rattles"