The story
1670s as verb, 1680s as noun. The origin is unknown, possibly from London street slang; ostensibly as *bant + -er (frequentative suffix). Possibly an Anglo-Gaelicism from the Irish bean (“woman”), so that "banter" means "talk of women."
Etymology from Wiktionary, CC BY-SA 4.0.