Part of the problem is that Scottish and Irish music have more, in terms both of repertoire and style, in common with English music than they do with, for example, Breton music, which in turn has more in common with French music, etc. etc.  The same is true of the general cultures of those countries.

As a person of mixed race (in fact, almost everybody over here is, whether they realise it or not), I have never understood this apparently irrational insistence on trying to impose what can only be an arbitrary dividing line between the traditional musics of the various nations that comprise what I may as well call, for convenience, the British Isles.  Is a song "Celtic" because it's sung in Scotland, or is it "English" when it's sung a hundred yards or so over the border?  Am I, as a person born in England of (recent) Scottish descent, more, or less, Celtic than a person born in Scotland of (common example) Italian parents?  In what way is such a distinction meaningful?

The "Celtic" label as we know it today began as a romantic -and rather ill-informed- notion of the 19th century, and is perpetuated mostly as a marketing ploy by the recording industry; except insofar as it applies to language and to archaeology, the term is so abused as to be functionally meaningless; we no longer talk about "Aryan" music, after all.  I have to admit that, like others, I describe the music I play as "Celtic" when answering questions from people who know nothing about the subject; they don't know what it means, but they think they do, so it saves time.