The story
* Relates to pointing with the finger (see 1880 citation), but sources differ: perhaps pointing to the empty space where the meat/fish would be. * Perhaps from point as the punctuation mark (period/full stop) marking termination.
Etymology from Wiktionary, CC BY-SA 4.0.