Etymology
Today it means: In the manner of a Puritan; puritanically. Full definition →
From Puritan + -ly.
Etymology from Wiktionary, CC BY-SA 4.0.