There seems to be an element of "damned if you do and damned if you don't". The early collectors have been criticised for being highly selective and only recording those songs which they considered to be 'proper' folk songs. Later collectors saw the importance of trying record a singer's entire repertoire without putting value judgements on the material. The purpose of the Roud Index, if I understand correctly, is to identify songs found in the oral tradition. That necessarily includes songs which which had clearly only recently entered the oral tradition from the stage. However even those often show variations between different singers - at what point do these slight variations become sufficient for them to have undergone the transformation required to become a 'folk song'?

As for them serving a different purpose, whilst Walter Pardon apparently saw a difference between different parts of his repertoire, I wonder whether the same was true of his audience? I suspect for them the purpose of his songs, whether folk songs or not, was simply to provide entertainment on a Saturday night in the pub.

As Jim correctly says, the oral tradition is an incredibly complex subject. Traditional singers performed material from many different sources, sometimes only to satisfy their audience but in other cases because they genuinely liked the songs. It appears to me that definitions should be used for guidance rather than to exclude. I am sure that most scholars are able to make appropriate distinctions depending on what aspects of the tradition they are studying.