The story
In Pennsylvania, on one side (shore) of the West Branch Susquehanna River, it was first settled by a family from New Jersey; settlers on the other side of the river referred to Waynesburg as the Jersey shore, and the name stuck.
Etymology from Wiktionary, CC BY-SA 4.0.