It is just a silly figure of speech (we have the same in German though we can't compare the apple to the pear which doesn't make it any better) that should be used less often. What often is meant is 'you can't equate apples and oranges'. The expression is too often used by persons who don't like a comparison and are too lazy to argue why they don't like it.

Of course you can compare apples and oranges, it is only a question of the tertium comparationis (= the third part in a comparison), that is in which respect they are compared. The tertium comparationis may be useful (nourishment value; preservability) or less useful (reflectivity; nose lengths). In any such situation you should argue whether the tertium comparationis is useful or not and why you think so and not replace argumentation and thinking by a silly figure of speech.

Wolfgang