The story
Probably by analogy with considering cap, which is attested from the early 17th century.
Etymology from Wiktionary, CC BY-SA 4.0.
thinking cap
think·ing capToday it means: A metaphorical state of focused thinking or concentration, invoked to encourage problem-solving. Full definition →