Skip to content

tokenize

Today it means: To reduce to a token or set of tokens by lexical analysis. Full definition →

Ask AI about its historyChatGPT Claude Perplexity Google AI Grok

The story

From token + -ize.

Etymology from Wiktionary, CC BY-SA 4.0.