The story
Traditionally pub names had a pictorial representation, and the inn sign for inns named "Five Alls" showed five representative figures, each in appropriate garb with a motto underneath. This has occurred in a number of variations, but usually included a monarch saying "I rule (for) all" or "I govern all", a bishop or minister saying "I pray for all", a soldier saying "I fight for all", and a farmer saying "I pay for all". (In modern signs, often John Bull pays for all.) The fifth figure was often a lawyer saying "I plead for all", or could be omitted, in which case the inn would be named "Four Alls".
Etymology from Wiktionary, CC BY-SA 4.0.