The story
With the adoption of decimal US currency in 1794, there was no longer a US coin worth one eighth of a dollar, but two bits retained the meaning of one quarter dollar. Because there was no one-bit coin, a dime (10¢) was sometimes called a short bit and 15¢ a long bit.
Etymology from Wiktionary, CC BY-SA 4.0.