How-to · Updated 2026-10-19
LLM context window too long: six ways to shrink it
When an LLM context window is too long, you either hit a hard length error or pay for tokens the model barely uses. Shorten it by trimming history, retrieving fewer chunks, capping tool output, clearing stale turns, caching a stable prefix, and compressing bulky context against the question being asked.
Why is my context too long?
Four sources account for most long prompts: conversation history that grows every turn, retrieved documents stuffed in whole, tool results and file reads, and large static instructions. Find which one dominates before you pick a fix. Log the token count of each section of the prompt for a few real requests.
1. How do you trim conversation history?
Drop or summarize old turns, or start a fresh session when the task changes. In Claude Code, /clear resets history and /compact summarizes it. See compact vs clear vs compression. Summaries are lossy, so keep any exact values you will need.
2. How do you retrieve less?
Return fewer chunks, rerank before you stuff, and set a token budget for retrieved text. When chunks are long and only a few sentences matter, compress them against the user's question, see context compression tools compared.
3. How do you cap tool and file output?
Paginate, filter and truncate with a clear marker, as covered in reducing tool call output tokens. Read file ranges instead of whole files.
4. How does prompt caching help?
Caching does not shorten the context, but it makes a stable prefix cheaper on repeat requests. Keep static instructions first and changing content last. Check your provider's current caching rules, because they differ.
5. How does question-aware compression help?
SuperCompress takes a bulky block and the current question, and the hosted Neural Keep model keeps the original lines that answer it, verbatim. Use it on logs, diffs, retrieved documents and MCP results before they enter the prompt. Output reports original_tokens and kept_tokens per call so you can measure it on your traffic.
6. What if the input is still too long?
Split the task. Process documents in parts, with an explicit question per part, and combine the answers. Larger context windows help, but long inputs still cost per token and models can attend poorly to material buried in the middle, so a shorter relevant prompt is often better than a longer complete one.
Frequently asked questions
How do I fix a context length exceeded error?
Reduce the largest section of the prompt: trim history, retrieve fewer chunks, cap tool output, or compress bulky context before sending it.
Does a bigger context window solve the problem?
It removes the hard error but not the cost, since you pay per input token, and very long prompts can bury the relevant part.
Is summarizing the same as compression?
No. Summarizing rewrites text and can lose exact values. Extractive compression keeps original lines and drops others.
Can SuperCompress fix a context length error?
It can shorten bulky inputs such as logs and documents so they fit, by keeping only lines relevant to your question. It does not change the model's window.