The story
Compare whale (“something or someone excellent”); thus, not just any good time but rather a great time.
Etymology from Wiktionary, CC BY-SA 4.0.
have a whale of a time
Today it means: To enjoy oneself greatly: to have a very good time. Full definition →