I don't think most of us would eat dirt intentionally, but I think we all know it when we get a salad with lettuce or spinach that wasn't washed well enough, etc. Or when the tap water in some new place has that taste. You can probably also attribute other aspects of the taste and texture from the fish's habitat to that "muddy" taste.

SRS