Token Counter
Split text the way a language model does and count the pieces. Useful for staying inside a context window or estimating what a prompt will cost, before you send it anywhere.
The pieces appear here as you type.
For the GPT tokenisers the splitting patterns are the published pre-tokeniser regexes from OpenAI's tiktoken, which is what decides where one piece can never join the next; those boundaries are exact. What this page cannot do is run the byte-pair merges inside a piece, because that needs the vocabulary file, which is over a megabyte and is not mine to ship. So a long or unusual word is estimated from its length and shape instead, which is why every token figure here carries a tilde. On ordinary English or German prose the total lands within about ten per cent of the real count; on minified code, long identifiers or emoji it drifts further. Claude is a rougher estimate again: Anthropic does not publish a vocabulary for Claude 3 and later, so that mode is modelled on its observed behaviour rather than on a specification. The SentencePiece and Gemma modes are likewise modelled on how those tokenisers are known to behave, not on their vocabularies. If you need an exact number, run tiktoken locally, or ask the provider's API to count for you.