According to Encarta: Swaddie (solder) [Early 19th century. Formed from a dialectal word used as an insult for a country person, of uncertain origin: perhaps from a Scandinavian source.]
The word 'swedebashers' is used as a derogatory term for farmers/peasants in the UK. Could there be a link?