The Crusades are being over-emphasized here. Musical ideas travelled before and after. Trade, diplomacy and mercenary service all figure in to it. Much was also disseminated through the Byzantine Empire (Constantinople was the most cosmopolitan city in the world) and Sicily.
Ideas also travelled back along the same route.
Gypsies were admired as novelty performers for the Ottoman Court, but the Sultan had his own highly trained musicians. The courtly music was a very sophisticated affair and not just anyone, no matter how good, could play it.
I don't quite remember about the Balkans, but for the rest of the Empire, it's hard to say how much of an influence Gypsies were on music. Some itinerant performers were Gypsy, or rather Travellers, but by no means all. Nor were most of the wandering minstrels.