Yes, it would properly be "depending on whom you ask", since "whom" is the object in that case, and "you ask" is not the object. The again, it would properly be "depending on who does it". That's because in that case the object of the prepostion "on" is an entire phrase "who does it", so the word "who" in that entire phrase remains unchanged. The word "who" is the subject of the phrase "who does it", while the phrase itself is the object of the preposition "on".

Confused yet? ;-)