# Dspy Embedding Retrieval

> Use for DSPy retrieval with dspy.Embedder, dspy.Embeddings, FAISS indexes, semantic search, and local or hosted embedding models.

- **Type:** Skill
- **Install:** `agentstack add skill-omidzamani-dspy-skills-dspy-embedding-retrieval`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [OmidZamani](https://agentstack.voostack.com/s/omidzamani)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [OmidZamani](https://github.com/OmidZamani)
- **Source:** https://github.com/OmidZamani/dspy-skills/tree/master/skills/dspy-embedding-retrieval

## Install

```sh
agentstack add skill-omidzamani-dspy-skills-dspy-embedding-retrieval
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# DSPy Embedding Retrieval

## Goal

Build semantic retrieval over an application-owned text corpus with `dspy.Embedder` and `dspy.Embeddings`.

## Basic Hosted Embedder

```python
import dspy

corpus = [
    "DSPy programs are composed from modules.",
    "MIPROv2 optimizes instructions and demonstrations.",
    "RLM explores large contexts with a sandboxed REPL.",
]

embedder = dspy.Embedder("openai/text-embedding-3-small")
search = dspy.Embeddings(corpus=corpus, embedder=embedder, k=2)

result = search("Which optimizer tunes prompts?")
print(result.passages)
print(result.indices)
```

## Use in RAG

```python
class LocalRAG(dspy.Module):
    def __init__(self, retriever):
        super().__init__()
        self.retriever = retriever
        self.answer = dspy.ChainOfThought("context: list[str], question -> answer")

    def forward(self, question: str):
        context = self.retriever(question).passages
        return self.answer(context=context, question=question)
```

## Custom Local Embeddings

Wrap any callable that accepts `list[str]` and returns a 2D numeric array:

```python
from sentence_transformers import SentenceTransformer
import dspy

model = SentenceTransformer("sentence-transformers/static-retrieval-mrl-en-v1")
embedder = dspy.Embedder(model.encode)
search = dspy.Embeddings(corpus=corpus, embedder=embedder, k=5)
```

## Scores, FAISS, and Persistence

Use `dspy.EmbeddingsWithScores` when downstream logic needs similarity thresholds or reranking.

For corpora at or above the `brute_force_threshold` default of `20_000`, DSPy builds a FAISS index. Install FAISS first:

```bash
pip install faiss-cpu
```

Persist the index when embedding the corpus is expensive:

```python
search.save("./retrieval-index")
loaded = dspy.Embeddings.from_saved("./retrieval-index", embedder=embedder)
```

## Related Skills

- Build a complete pipeline: [dspy-rag-pipeline](../dspy-rag-pipeline/SKILL.md)
- Design typed context fields: [dspy-signature-designer](../dspy-signature-designer/SKILL.md)
- Harden caches: [dspy-production-deployment](../dspy-production-deployment/SKILL.md)

## Best Practices

1. Evaluate retrieval quality separately from answer quality.
2. Keep corpus chunking deterministic and versioned.
3. Persist expensive indexes.
4. Use `EmbeddingsWithScores` when debugging relevance.
5. Measure memory and latency before enabling FAISS for large corpora.

## Official Documentation

- **Embedder API**: https://dspy.ai/api/models/Embedder/
- **Embeddings API**: https://dspy.ai/api/tools/Embeddings/

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [OmidZamani](https://github.com/OmidZamani)
- **Source:** [OmidZamani/dspy-skills](https://github.com/OmidZamani/dspy-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-omidzamani-dspy-skills-dspy-embedding-retrieval
- Seller: https://agentstack.voostack.com/s/omidzamani
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.