Both Wiktionary (on the word 'tarrier') and Wikipedia (on the song under discussion) refer to 'tarrier' as a nickname for Irish workers; since the nickname is listed as used in Northern Ireland and Scotland it's probably pejorative.