It wasn't a music box, it's a tune being played on a barrel organ by one of the street musicians in the game. I think there's a few places where he can be found. Trafalgar Square is usually where I see him.