Distinguishing between Muslims and non-Muslims by what they are wearing would be extremely difficult where I live. Especially in the case of men.   About as easy as telling Catholics from Protestants.