Minor point of fact, Ladybird: The United Empire Loyalists were refugees from the American Revolution, not the U.S. Civil War (although the Revolution was in fact a kind of civil war itself).
Many of the people who emigrated to Canada from whatever country (many of my own forebears among them) came to avoid getting involved in wars they didn't believe in, or because they backed the wrong side and faced persecution for it.
Also, I think Mike Regenstreif is correct that the Canadian government, to its credit, didn't make it particularly difficult for draft evaders to come to Canada. In fact, Canada didn't really support the war at all, and drew criticism from the U.S. government for it. Some Canadians fought for the U.S., but only as private citizens, with no encouragement from their government.
A number of my close friends, many of them musicians, came to Canada via the draft evasion route. I think the U.S.'s loss was our gain.