There seem to be several theories. Here's one from BBC h2g2

Although the phrase 'cock-up' might appear to have come about in a similar way to 'balls-up', its origins are actually in beer making. If the batch went bad, they turned the cock (ie tap, or faucet) up to drain the barrel. However, the word 'cock', a Middle and Old English word, is one of the many vulgarities for the penis. In London, though, Cockneys appear to have both terms in mind when they say 'Wotcher cock', which comes from the term 'cock sparrow' (pronounced 'sparrah'). It is a general term for a man, although 'cock sparrow' was usually saved for small boys. It has been used for about 300 years.

And, from Michael Quinon's excellent website, World Wide Words:

Oddly, in British English it is not these days a vulgarism, or at least only a very mild one. It comes from one of several senses of cock, to bend at an angle, as in—for example—cocking a gun or turning up the brim of one's headgear (so producing an old-time naval officer's cocked hat).

The use of cock-up to mean a blunder or error was originally British military slang dating from the 1920s. The slang sense of cock clearly had a lot to do with its adoption, but this hasn't stopped it being used in respectable publications, and modern British dictionaries mark it merely as informal or colloquial.

The longer phrase I used it in, "a cock-up on the [something] front" was coined in a BBC television comedy The Fall and Rise of Reginald Perrin some 20 years ago and has become a minor catchphrase. The original was "there's been a bit of a cock-up on the catering front", which was spoken by a former army officer, not over-blessed with savvy, who was totally confused by civilian life and had either forgotten to buy any food, or run out of money to do so.

[I'm indebted to Nigel Rees for confirming the provenance of this catchphrase.]