Green men are also a common motif in folklore. Connected to magic, forest, pagan religion. Maybe the term little green men refers to this too, green men being something the superstitious believe in.
Letty