So I can't answer why it's Jack of Diamonds in specific but it was sung by railroadmen who had lost money playing conquian, so Wikipedia tells me, but I can't find anything all that special about Jacks or Diamonds in relation to that game.

According to this card website, "All Jacks correspond with number 11. By some called the number of Evil because it transcended the number of commandments, and did not attain 12, the number of Grace and perfection.... The Jack of Diamonds win their crown through good guardianship of values. Jack of Diamonds is a controversial card. They can be wholly material, highly spiritual, or curios and unaccountable, or a mixture of two. They are aware that hard and stable efforts are a requirement for success. Jack of Diamonds are mentally sharp and clever, and always looking for new enterprises, ventures and opportunities to make their kingdom, but spiritual confusion can lead them to drift through life and live by their wits, usually managing to get by....   Fixation on material matters can lead to a loss of spirituality. Diamonds are believed to be the suit of money, but it is a suit of values - and our values include both - material, spiritual, moral values of human beings."

http://www.sevenreflections.com/cards-of-destiny/cards/jack-of-diamonds/

According to tarrot cards, "It is widely accepted that the Jack of Diamonds' nature is selfish, jealous, unreliable and possibly dishonest. On those lists where his character is not described, this Jack is presented as someone who brings bad news. Two of the entries above present him as a relative of the inquirer's. The meaning that best covers his attributes is that which reads, 'a relative; someone not really reliable, selfish or jealous; bad news'. There are really three meanings here: the card may indicate either a male relative, or someone not very reliable, perhaps selfish or jealous of the querent, or bad news – or more accurately the bearer of bad news. In a reading the card can indicate any one of these three things, and sometimes permutations of the same, as when a male relative is the bearer of bad news."

Additionally, the ten of diamonds is apparently the card of money, and the queen of diamonds is "A fair woman, fond of company and a coquette." So gamblers are naturally constantly between money and women, too.

https://auntietarot.wordpress.com/2012/01/07/playing-cards-diamonds-1/

At least that's what they say.