The Whigs were the antecedents of the nineteenth century Liberal party (along with the Peelites), made up mainly of the great artistocratic families and their relatives. You couldn't become a Whig in the modern political sense (although you could support them by voting for their candidates) - you either were one by birth or you were not. The 'blues' would be the Tories, who became the modern Conservative Party (it's still their political colour). In effect, the Tories were the party of the gentry, the country squire etc.

There probably is a connection within the song to the American Revolution - its liberal/radical ideas would always find a more sympathetic hearing among the Whigs and their supporters than the Tories.