Search optimization guide
Token compression for AI search
LLM-enhanced search sends product data, filters, and user context with every query. Compression keeps only the search-relevant product attributes, cutting costs and improving relevance.
Compressed search pipeline
from supercompress import Compressor
comp = Compressor()
def search_products(query, product_catalog):
# Compress the product catalog against the search query
result = comp.compress(product_catalog, query)
# Only products matching the query intent remain
return llm.generate(
f"Search results for '{query}':\n{result.compressed_text}"
)Frequently asked questions
Does compression slow down search?
Compression adds ~60ms, but the LLM generates faster with less context. Net impact is neutral or faster.
Can I use it with Elasticsearch or Algolia?
Yes. Use SuperCompress as a re-ranking step after your search results are returned.
Build with less context
Put compression in front of your next LLM call.
Use the hosted API or run SuperCompress locally. Keep the evidence, drop the token waste, and measure the savings before it reaches your model.