Framework integration
Haystack compression integration
Haystack provides modular NLP pipelines. Add SuperCompress as a custom component that compresses documents before generation.
Haystack custom component
from haystack import component
from supercompress import Compressor
@Component
class SuperCompressPreProcessor:
def __init__(self):
self.comp = Compressor()
@component.output_types(documents=list[Document])
def run(self, documents: list[Document], query: str):
compressed = []
for doc in documents:
result = self.comp.compress(doc.content, query)
doc.content = result.compressed_text
compressed.append(doc)
return {"documents": compressed}Frequently asked questions
Where in the pipeline should I add compression?
After the retriever and before the generator. This compresses retrieved documents before the LLM prompt is built.
Does it work with Haystack 2.x?
Yes. The component is compatible with Haystack's new component API.
Build with less context
Put compression in front of your next LLM call.
Use the hosted API or run SuperCompress locally. Keep the evidence, drop the token waste, and measure the savings before it reaches your model.