What is a context window in AI?

The context window is the maximum amount of text an AI model can take into account at one time. It is like the model's working memory. Everything the model considers when writing a reply, including your instructions, the conversation so far, any uploaded documents and the reply it is currently writing, has to fit inside this window.

Context windows are measured in tokens, the small chunks of text that models process. A token is roughly three-quarters of an English word. Early chatbots had context windows of only a few thousand tokens, enough for a few pages. Many modern models handle hundreds of thousands of tokens, and some handle a million or more, which means they can read entire books, large codebases or long reports in one go.

Why does it matter?

Long documents. A larger context window lets you upload a full annual report or contract and ask questions about the whole thing, instead of cutting it into pieces.

Long conversations. Every message in a chat is sent back to the model with each new turn. When a conversation outgrows the window, older parts must be dropped or compressed, which is why chatbots sometimes forget earlier instructions.

Coding. Developers can give the model many files at once, so it understands how different parts of a project connect.

Cost and speed. With APIs, you pay for every token in the context, so filling a huge window with every request gets expensive and slower.

A bigger window is not automatically better. Research and practical experience show that models can pay less attention to information buried in the middle of very long inputs. A model might technically accept 500 pages but still overlook a detail on page 230. Quality of attention matters, not just size.

Some practical tips help:

1. Give the model only what is relevant to the current question.
2. Put your most important instructions near the end of a long prompt, close to the question.
3. Start a new chat when switching topics.
4. For huge document collections, use retrieval methods (RAG) to fetch only the relevant sections.

Remember too that the context window is not the same as long-term memory. Once a conversation ends, the model does not retain it unless the app has a separate memory feature that saves information across chats.