The story
From to (“indicating a limit reached”) + a + man (“adult male human; human regardless of gender or sex”), indicating that something reaches the extent of every individual person.
Etymology from Wiktionary, CC BY-SA 4.0.
to a man
Today it means: Including every person; without exception; unanimously. Full definition →