The story
From the phrase cold enough to freeze the balls off a brass monkey - supposedly a reference to the brass container for cannon balls on a British man-of-war (but this seems to have been discredited).
Etymology from Wiktionary, CC BY-SA 4.0.
brass monkeys
Today it means: Very cold. Full definition →