If you start looking at the history of costume (that's costume as in clothing - not theatrical costuming) it's interesting that men's clothing was for the most part as colorful and varied as womens - or more so until the late Victorian era. so it's really only been a few generations.

I've seen speculation that the trend towards monotony and less flamboyance was more due to the widespread involvement of men in the military for the first half of the 20th century.