Chippenham & Bradford-on-Avon both seem to have a lot of folky things going on. Chippenham folk festival is really good.

If you're looking for an interesting small town with a nice community, lots of history & a decent High St. I'd suggest Malmesbury. It's about 15 mins drive from Chippenham, and about 30 mins from Bradford-on-Avon, Swindon, Bath & Bristol - but like a lot of Wiltshire the public transport's not very good. The Womad festival takes place every year just outside of Malmesbury and residents get cheap tickets, which is nice if it's your sort of thing.