Hi Ed - "Coda" in Edinburgh has a good selection of folk and traditional CDs. It's on "The Mound", I think, but you can get a proper address if you "google " it. As Megan says, there is a shop with a fair selection of CDs in the Victorian market in Inverness. As far as I know ,it's still there. My wife was in Kirkwall 2 days ago and told me the shop there which used to have quite a good selection of stock appeared to be closed, but there may have been a reason for that.
If you're looking specifically for LPs, try some of the many charity shops, "Oxfam" is sometimes quite good, although you could be buying a lot of scratches as well. Make sure you check any discs first.
Best of luck, Kenny