Best practices
Common prompt compression mistakes
Prompt compression is straightforward to implement but easy to get wrong. Here are the most common mistakes teams make and how to avoid each one.
Mistake 1: Compressing system prompts
System prompts contain instructions that shape the model's behavior. Compressing them can remove critical behavioral instructions. Fix: Always preserve system prompts. Only compress user-provided context and conversation history.
Mistake 2: Using the wrong compression budget
A compression budget of 50% might sound safe, but for short contexts, 50% removal could drop important content. Fix: Use absolute token budgets for short contexts and percentage budgets for long contexts.
Mistake 3: Not testing with real queries
Testing compression with generic queries like "Summarize this" hides quality issues. Fix: Test with the actual queries your application receives in production.
Mistake 4: Compressing every call
Not every LLM call needs compression. Very short prompts (under 500 tokens) see minimal benefit. Fix: Only compress when the context exceeds 1,000 tokens.
Mistake 5: Ignoring compression logs
SuperCompress returns token counts and compression ratios. Fix: Log these metrics and monitor for unexpected changes in compression behavior.
Frequently asked questions
What is the most critical mistake?
Compressing system prompts. This can break your application's behavior entirely.
How do I monitor compression quality?
Log the compression ratio, oracle recall estimate, and query for every compressed call.
Build with less context
Put compression in front of your next LLM call.
Use the hosted API or run SuperCompress locally. Keep the evidence, drop the token waste, and measure the savings before it reaches your model.