Millard Fillmore "chose to retire". Not exactly. He ran again in 1856. By 1855 he was trying to inveigle Know-Nothings into the Whig party, since he realized they were in fact the only booming party at the time, and signing up many former Whigs. After trying to get Know-Nothing support without actually joining them, he realized it wouldn't work and did join them. Still didn't work for him--he was never elected president.

There's a lot more to the story--regarding the Fugitive Slave Act, election of 1852 (Whigs decided to run Winfield Scott, since they had won with William Henry Harrison and Zachary Taylor--in fact they only won with generals--- and the third time was not lucky)--- and other topics. I have more of the story in my post of 8 May 2004 10:36 PM, but there's yet more.