The story
A reference to the fact that the speaker is about to say something blunt rather than fine (that is, delicate or subtle).
Etymology from Wiktionary, CC BY-SA 4.0.
not to put too fine a point on it
Today it means: Used to apologize for a possibly impolite statement one is making; to put it bluntly; to express it in plain terms. Full definition →