The story
From the practice of classifying agricultural commodities as fair or middling. The earliest attested instance of these descriptions being combined into the phrase fair to middling was in a British publication of 1822 but the first instance of this usage being generalised is from an American publication of 1837
Etymology from Wiktionary, CC BY-SA 4.0.