The story
From the late medieval custom symbolizing a challenge, in French jeter le gant which nowadays usually means to give up after having tried hard (leaving the challenge to any takers). The English phrase dates to the 1540s (figurative use by the 18th century).
Etymology from Wiktionary, CC BY-SA 4.0.