I've always liked the "we've been there ten thousand years" verse because it captures precisely the notion of an infinite set, which was long deemed absurd by many mathematicians: that it contained proper subsets that were as big as the set. Georg Cantor (who would probably have not felt at home in a Christian hymn) developed his theory of such sets starting with just that characteristic.