The story
In reference to the time a person can work on their garden during their paid leave.
Etymology from Wiktionary, CC BY-SA 4.0.
garden leave
Today it means: A period of paid leave before termination of employment, often including non-compete contract clauses. Full definition →