M. Ted wrote, "It always struck me that the song was really "St. James Infirmary"" . . .

Interesting comment. "Summertime" and "St. James Infirmary" are both minor blues, so your observation has a lot of truth, as all blues are built around the same basic formula (i.e. 1-4-5 riff, which as Chet W. pointed out above, are Dm-Gm-Am for this tune. I play both of these on harmonica in what they call 3rd position, which is well suited for minor blues.) There are differences between the two of course - St. James has an intro and outro verse, and I think of it as a "hot jazz" tune, while Summertime is a slow, smouldering vocal. But yeah, they're the same riff in many ways.