# Haystack

> >-

- **Type:** Skill
- **Install:** `agentstack add skill-magnus919-agent-skills-haystack`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [magnus919](https://agentstack.voostack.com/s/magnus919)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [magnus919](https://github.com/magnus919)
- **Source:** https://github.com/magnus919/agent-skills/tree/main/haystack

## Install

```sh
agentstack add skill-magnus919-agent-skills-haystack
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Haystack Expert Skill

Haystack (by deepset) is a production-oriented framework for building search and NLP pipelines. Its core abstraction is the **Pipeline** — a directed acyclic graph of typed components with explicit connections. Unlike LangChain's LCEL (pipe operator) or LlamaIndex's query engines, Haystack pipelines are **declared upfront with add_component and connect**, giving validated, debuggable DAGs.

## Core Paradigm

```python
from haystack import Pipeline
from haystack.components.embedders import SentenceTransformersTextEmbedder
from haystack.components.retrievers.in_memory import InMemoryEmbeddingRetriever
from haystack.components.builders import PromptBuilder
from haystack.components.generators import OpenAIGenerator
from haystack.document_stores.in_memory import InMemoryDocumentStore

# Build a pipeline
document_store = InMemoryDocumentStore()
pipeline = Pipeline()
pipeline.add_component("embedder", SentenceTransformersTextEmbedder())
pipeline.add_component("retriever", InMemoryEmbeddingRetriever(document_store=document_store))
pipeline.add_component("prompt_builder", PromptBuilder(template="Answer using: {{documents}}\n\nQuestion: {{question}}"))
pipeline.add_component("generator", OpenAIGenerator())

# Connect components
pipeline.connect("embedder.embedding", "retriever.query_embedding")
pipeline.connect("retriever.documents", "prompt_builder.documents")
pipeline.connect("prompt_builder", "generator")

# Run
result = pipeline.run({"embedder": {"text": "What is Haystack?"}, "prompt_builder": {"question": "What is Haystack?"}})
```

## Core Principles

1. **Pipelines are validated DAGs.** add_component + connect. Pipeline validation catches errors BEFORE execution — leverage this during development.
2. **Components are typed.** Each component has input/output slots. Connections must match types. This prevents runtime errors.
3. **PromptBuilder uses Jinja2.** Templates are Jinja2 strings, not f-strings. `{{documents}}`, `{{query}}`, `{{question}}` are variable placeholders.
4. **Indexing and query are separate pipelines.** One pipeline loads/cleans/embeds/writes documents. Another retrieves/generates answers. They share the DocumentStore.
5. **Evaluation is a pipeline too.** Add evaluator components to measure faithfulness, relevancy, or custom metrics.

## Where to Start

| You already have... | Start here |
|---|---|
| Nothing — exploring Haystack | Build a basic indexing + query pipeline |
| Documents to index | Build an indexing pipeline (converters, splitter, embedder, writer) |
| A search use case | Build a query pipeline (embedder, retriever, prompt, generator) |
| A production deployment | Add Hayhooks + evaluation pipeline |

## Quick Reference

| Task | Approach | Reference |
|------|----------|-----------|
| Build indexing pipeline | add_component -> connect -> run | `references/pipeline-design.md` |
| Build query pipeline | retriever -> prompt_builder -> generator | `references/pipeline-design.md` |
| Choose document store | InMemory (dev), Elasticsearch/Pinecone (prod) | `references/document-stores.md` |
| Embedding retrieval | SentenceTransformersTextEmbedder + EmbeddingRetriever | `references/retrievers.md` |
| Hybrid retrieval | BM25 + Embedding in parallel, DocumentJoiner | `references/retrievers.md` |
| Prompt templates | Jinja2 in PromptBuilder | `references/pipeline-design.md` |
| Evaluation | DeepEvalEvaluator, SASEvaluator | `references/evaluation.md` |
| Deploy | Hayhooks REST API | `references/deployment.md` |

## Framework Routing Guide

| Scenario | Reach for | Why |
|----------|-----------|-----|
| Search / NLP pipelines | **Haystack** | Pipeline DAG model is most mature for retrieval-heavy workloads |
| Documents to query / RAG | **LlamaIndex** | Data ingestion is the primary primitive |
| Chain/agent composition | **LangChain** | LCEL pipe operator for general chain building |
| Compiled prompt programs | **DSPy** | Auto-optimizes prompts against a metric |
| Role-based multi-agent | **CrewAI** | Higher-level agent abstraction |

## Reference Files

| Reference | Load when | File |
|-----------|-----------|------|
| Pipeline Design | Building indexing and query pipelines | `references/pipeline-design.md` |
| Document Stores | Store selection and configuration | `references/document-stores.md` |
| Retrievers | Embedding, BM25, hybrid retrieval | `references/retrievers.md` |
| Validation Audit | Research validation of all API claims | `references/validation-audit.md` |
| File Converters | Multi-format indexing, YAML serialization, component types | `references/file-converters.md` |
| Evaluation | Metrics, evaluators, pipeline evaluation | `references/evaluation.md` |
| Deployment | Hayhooks, containerization, production | `references/deployment.md` |
| FAQ & Troubleshooting | Common errors and fixes | `references/faq-and-troubleshooting.md` |

## Templates

| Template | When to use | File |
|----------|-------------|------|
| Indexing Pipeline | Load, split, embed, write to store | `templates/indexing-pipeline.py` |
| Query Pipeline | Retrieve, prompt, generate answer | `templates/query-pipeline.py` |
| Hybrid RAG | BM25 + embedding in parallel | `templates/hybrid-rag.py` |

## Troubleshooting

| Symptom | Likely cause | Fix | Reference |
|---------|-------------|-----|-----------|
| Pipeline run errors | Component connection mismatch | Check component input/output slot types | `references/pipeline-design.md` |
| No documents retrieved | Empty document store | Run indexing pipeline first | `references/pipeline-design.md` |
| Prompt not rendering | Wrong variable name in Jinja2 template | Check {{variables}} match pipeline input | `references/pipeline-design.md` |
| Slow retrieval | Full scan instead of ANN | Configure approximate nearest neighbor index | `references/retrievers.md` |
| Embedding mismatch | Different models for indexing vs query | Use same model in both pipelines | `references/retrievers.md` |
| Hayhooks not starting | Port conflict or missing config | Check port, run with --help for options | `references/deployment.md` |

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [magnus919](https://github.com/magnus919)
- **Source:** [magnus919/agent-skills](https://github.com/magnus919/agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-magnus919-agent-skills-haystack
- Seller: https://agentstack.voostack.com/s/magnus919
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.