The story
The term “one-horse” originated as an agricultural phrase, meaning to be drawn or worked by a single horse. This led to the use of this phrase in a metaphorical sense as something that is small or insignificant. Charles Dickens explained in his publication All the Year Round (1871): ‘One horse’ is an agricultural phrase, applied to anything small or insignificant, or to any inconsiderable or contemptible person: as a ‘one-horse town,’ a ‘one-horse bank,’ a ‘one-horse hotel,’ a ‘one-horse lawyer’, etc.
Etymology from Wiktionary, CC BY-SA 4.0.