This link will take you to a summary of a biography of Stephen Foster written by his brother and published in 1896. The summary gives background for a couple of songs, but unfortunately not Hard Times. The book is reported to have the background of 160+ songs, if anyone has a copy of it.

Hard Times is my favorite of Foster's songs I've heard so far. It can relate to anyone's hard times. It was a particular favorite of the Mormon pioneers (my heritage) and they made many versions of it, some serious and some parodies.

BTW, isn't it "FRAIL forms fainting by the door"? I always assumed this line referred to slaves, who certainly wouldn't have been "pale." I could be wrong.