TJ, you are entirely ON base; this is an example of singing "without using actual words."

I think that the only essential difference between "scat" and the kind of nonsense syllables found in this and many other folksongs is that the word "scat" usually refers to improvised passages, where both the syllables and the melody are made up on the spot for "one-time use," whereas a nonsense-syllable refrain is normally exactly the same (or essentially so, at least) at the end of each verse.