What is a context window?

A context window is, in Anthropic’s words, “all the text a language model can reference when generating a response, including the response itself.”^[1] The same documentation describes it as a “working memory” for the model, separate from the much larger body of data the model was trained on.^[1] Whatever you want the model to take into account on a given turn has to fit inside that window.

The window is measured in tokens, not words. A token is a chunk of text the model processes; OpenAI describes tokens as “commonly occurring sequences of characters” and offers a rough rule of thumb that “1 token is approximately 4 characters or 0.75 words for English text.”^[2] So a few hundred words of plain English is a few hundred tokens, and the page or two of text in a short note is roughly the same order. Counting in tokens matters because the limit is a token limit.