Token limit
A token limit is a cap on how much text can be processed in a request or a period. It shows up in three forms: the context window's size, a maximum output length per response, and rate limits on tokens per minute or per month for an account.
The three fail differently and get confused with each other. A context limit means input has to be dropped or summarised. An output limit means a long response is cut off mid-generation. A rate limit means the request is refused entirely until the window resets.
The one that surprises people most is the output cap, because a truncated file looks like the model losing the thread rather than a hard ceiling.
What this means in practice
When generated code stops mid-function, that is usually an output limit rather than confusion. Ask for one file at a time. When an agent forgets a requirement stated twenty messages ago, that is the context window, and a fresh conversation fixes it.
Prompts for the tools this applies to
Related terms
- Context window
The context window is the maximum amount of text a model can consider at once, measured in tokens and covering everything in the request: system prompt, your instructions, the files supplied, prior conversation, and the response being generated..
- Input tokens
Input tokens are the units of text a model reads: your prompt, the files supplied, the system prompt, and the conversation so far.
- Output tokens
Output tokens are the text a model generates: explanations, code, and — with reasoning models — internal thinking that is billed even when it is not shown.