I don't know that the site was so important to the Muslims until the detractors decided to make an issue of it. It is, however, one of the better sites for a religious building in Lower Manhattan - and there are lots and lots of other churches in the area to prove the point. If you go a few blocks north, you get into an out-of-the way district where nobody goes, and there's not enough sidewalk traffic to make people feel safe. To the south and east and west are skyscrapers full of offices and condominiums, not really conducive to a religious center. The buildings on Park Place are of moderate size (and probably moderate expense). I think the choice would be to stay out of lower Manhattan altogether - maybe that's what the opponents want.
In short, the building in question is an insignificant building on an out-of-the way side street. It isn't an "important" site at all, which is why I wonder why people are making such a fuss about it.

-Joe-