11 miles is just typical singing exaggeration. The number eleven (probably) has the right number of syllables to fit the rhyme scheme and is a reasonable yet long distance to walk each day.

While some people who are in very good shape can walk 20+ miles a day (even through rugged terrain), most people have there upper limits around 10-15, and doing this every day takes its toll on your average person.