Oxford is my favorite city so far; very cool. It might not be TOO crowded with tourists in May...definitely on the train.

You might like Salisbury as well. Smaller, very charming, very scenic, nice shops, close to Stonehenge, too...but you might not have time to get the bus there...