Yellow was (is?) the color of the U.S. Cavalry - just as Red is for Artillery and Blue is for the Infantry. The yellow ribbon ("Around her neck she wore a yellow ribbon, she wore it for her lover who was far, far away") meant that her lover was a cavalry officer.