Metrics guide
Measuring token compression
Not all token compression is equal. Understanding the right metrics helps you evaluate whether compression is actually saving you money without hurting quality.
Key metrics explained
| Metric | Definition | Target |
|---|---|---|
| Compression ratio | Tokens removed / original tokens | 60-85% |
| Oracle recall | % of answer lines preserved | 100% |
| Cost savings | Monthly $ saved | As high as possible |
| Quality delta | Answer accuracy change | ≥0% (no degradation) |
Frequently asked questions
What compression ratio should I target?
Start with 65%. Higher ratios may start to impact quality.
How do I measure answer quality after compression?
A/B test: run 100 queries with and without compression, compare answers for factual accuracy.
Build with less context
Put compression in front of your next LLM call.
Use the hosted API or run SuperCompress locally. Keep the evidence, drop the token waste, and measure the savings before it reaches your model.