How about coming to the Gower Folk Festival which I think is the 7th-9th June. This is a small but very nice festival which is held in The Gower Heritage Centre on the beautiful Gower Peninsular.It is bit further than the Welsh border I'm afraid, West Wales in fact. But you can easily get there if you have transport?