The story
The first citation in the Oxford English Dictionary is from the Dictionary of Forces' Slang (1948). The OED suggests that it derives ultimately from the noun cock, but gives no further detail. The nature of the earliest citation suggests that this expression entered the wider language from military slang, making etymologies from typesetting or archery (see alternate meanings of cock-up) unlikely. One proposal is that the phrase originated describing an arrangement of the yards of a square-rigged vessel in port. Modern folk etymology has sometimes suggested that "cock-up" refers to a male erection, or to the phrase "cacked up", but neither is supported.
Etymology from Wiktionary, CC BY-SA 4.0.