Best practices
When to use prompt compression
Prompt compression is not always the right answer. Here is a decision framework to help you determine when to compress and when to skip.
Compress when:
- Context exceeds 1,000 tokens — Below this, the savings may not justify the overhead
- Context contains irrelevant information — Standards table with update data for many questions, only some questions touch most standards
- Cost is a primary concern — If you are tracking LLM spending, compression is your highest-ROI optimization
- Latency matters — Compressing large contexts reduces LLM prefill time, often netting faster responses
Skip when:
- Context is under 500 tokens — Minimal savings, and the compression overhead may not be worth it
- Every token matters — Some critical applications need every line of context available, regardless of relevance
- You are debugging prompt quality — Compression adds a variable that complicates debugging
Frequently asked questions
Should I always compress RAG contexts?
Yes. RAG contexts are the highest-ROI target for compression due to their size and noise level.
Should I compress few-shot examples?
Not usually. Few-shot examples are carefully chosen and typically small. Compression is more valuable on large context blocks.
Build with less context
Put compression in front of your next LLM call.
Use the hosted API or run SuperCompress locally. Keep the evidence, drop the token waste, and measure the savings before it reaches your model.