Since my posting above has been quoted in Azizi's first link, it may be worth mentioning that Agee's book (_A Death in the Family_) has since appeared in a new edition, by Michael A. Lofaro (Univ. of Tennessee Press, Knoxville, 2007). It contains no new information about Rufus & Rastus, but it does include a chapter (about a visit to an amusement park) with further enlightenment about the terrible cost that institutional racism inflicted on decent white people. It also makes clear that the book, tho nominally fiction, was intimately autobiographical (Agee originally used the real names of his relatives) & so deserves additional respect as documentation. Whether or not you have read the original best seller, I recommend this edition highly.

There are some other passages along the same lines in Agee's earlier book _Let Us Now Praise Famous Men_.

It might be worthwhile to date the "asbestos" joke, but I do not remember when I heard it (probably the 1950s) or from whom.