Jargonautica
Language Models Specifically

Context Window

How much text a model can hold in view at once, counting both what you have sent and what it has replied. Once a conversation exceeds it, the earliest material falls out of view and the model genuinely no longer has access to it. This is why very long chats start losing track of things established at the beginning.

Technical view

The limit exists because attention cost grows with the square of sequence length, so doubling context roughly quadruples the compute required. Various techniques have pushed practical limits from a few thousand tokens to hundreds of thousands. A large window is not the same as reliable use of it: models often show a ‘lost in the middle’ effect, attending well to the beginning and end of a long input while under-weighting material in the middle. Nothing persists between separate conversations unless it is deliberately stored and re-supplied.


Part of AI Terminology Index, section: Language Models Specifically.

How it connects

Context Window links to 27 other terms, and the 12 closest are drawn here. The faint lines are links between those terms. See it in the full graph.

AttentionAttentionContext EngineeringContext EngineeringLost in the MiddleLost in the MiddleRetrieval Augmented GenerationRetrieval Augmented…TokensTokensCachingCachingTransformerTransformerMemoryMemoryContext Length ExtensionContext Length Exte…Positional EncodingPositional EncodingCache-augmented GenerationCache-augmented Gen…Prompt CompressionPrompt CompressionContext Window