Unsearch

UnSearch

> Verifiable web retrieval for AI agents. Every result signed, hashed, snapshotted, and replayable months later.

Apache 2.0. MCP-native. Self-hostable on Cloudflare Workers + Containers. WACZ-aligned signed citation envelopes so any retrieval can be replayed in court, in an FDA submission, or in an EU AI Act Article 12 audit log months after the fact.

[](https://pypi.org/project/unsearch/) [](https://www.npmjs.com/package/@unsearch/sdk) [](LICENSE)

---

The problem

AI agents confidently make claims tied to web sources that are wrong, dead, paraphrased, fabricated, or — worst — silently changed since the agent retrieved them. There is no widely-adopted infrastructure primitive that produces a cryptographically verifiable record of "what the agent saw, at what URL, at what time, and what hash" — so every team building agents for regulated workflows is rebuilding this primitive from scratch.

The numbers from 2026 so far:

• $145K in U.S. court sanctions in Q1 2026 alone for AI-hallucinated legal citations. Single largest: $110K on April 4 2026 (23 fabricated citations + 8 false quotations across three filings).
• Harvey AI ($8B valuation) still hallucinates 1 in 6 queries.
• 40–60% reference fabrication rate on biomedical questions without retrieval; even with RAG, accuracy is only 69.5%.
• EU AI Act Article 12 full enforcement begins August 2026 — automatic event logging required over the system's lifetime, 6-month log retention minimum, 10-year documentation retention, provenance documentation explicitly required. Penalty: €15M or 3% of worldwide turnover.

Regulated buyers cannot use Anthropic's native web_search or OpenAI Codex CLI search because the citations aren't customer-pinnable, the snapshots aren't reproducible, and the data leaves their perimeter. So they hand-roll: Tavily + Firecrawl + Playwright snapshots + custom NLI grader + Postgres provenance table. Five vendors and 1–2 FTEs of glue per company.

UnSearch is the missing primitive.

---

For whom

UnSearch is built for:

• Engineering leads at regulated-AI startups (Series Seed–B, 10–80 people) building vertical AI for legal / medical / finance / insurance / research / compliance verticals
• AI platform directors at established regulated companies (banks, hospital systems, insurance carriers, BigLaw firms, pharma) retrofitting LLM features ahead of EU AI Act enforcement
• Citation-integrity research & journalism engineers snapshotting cited sources so retraction-tracking and reproducibility survive bit rot

UnSearch is not for:

• Indie devs building non-regulated agents — Anthropic native web_search and Codex CLI search are free and good enough for them
• Anyone needing a consumer search UI (we're an API + MCP, not a browser)
• Anyone needing internal-document search across Slack/Drive/Confluence (that's Glean's job)
• Anyone needing aggressive scraping of login-walled or anti-bot sites (we respect robots.txt and ToS)

See [docs/strategy/icp.md](./docs/strategy/icp.md) for the three personas in detail.

---

How it works

Every search or extract call returns a signed citation envelope plus a content-addressable snapshot in your own R2 bucket (self-host) or ours (hosted). The verify_claim endpoint then takes any {claim, source_url} pair and returns span-level evidence with a confidence score.

// citation_envelope returned with every result
{
  "v": 1,
  "url": "https://example.com/article",
  "fetched_at": "2026-05-28T18:32:11Z",
  "content_sha256": "a3f5...",
  "content_type": "text/html",
  "snapshot_r2_key": "citations/a3f5.../snapshot.wacz",
  "engine": "searxng:google",
  "agent_run_id": "run_01HQ...",
  "api_key_id": "key_01HK...",
  "signature_hmac_sha256": "9d2e..."
}

Envelope format is aligned with the WACZ-Auth spec so any consumer of the broader web-archival ecosystem can verify our snapshots with existing tooling. Full schema: [docs/citation-envelope.md](./docs/citation-envelope.md).

---

Quick start — MCP-first

The fastest way to evaluate UnSearch — no signup, no API key, no credit card. Free tier 5,000 verified searches / month is built into the MCP server itself:

claude mcp add unsearch
# or, in any other MCP-compatible client:
npx @unsearch/mcp-server

Then ask Claude (or any MCP-compatible agent):

> Search for "Q1 2026 legal AI sanctions" and verify the claim that the single largest sanction was $110,000 against the URL you find.

You'll get back signed results and a verify_claim response with evidence spans.

Or use the SDK

# Python
pip install unsearch

from unsearch import UnSearch

client = UnSearch(api_key="uns_...")
hits = client.search({"query": "EU AI Act Article 12 logging requirements"})
for r in hits["results"]:
    print(r["rank"], r["title"], r["citation_envelope"]["content_sha256"])

# Verify a claim
verify = client.verify_claim({
    "claim": "EU AI Act full enforcement begins August 2026",
    "source_url": hits["results"][0]["url"],
})
print(verify["supported"], verify["confidence"], verify["evidence_spans"])

# TypeScript / Node / Bun / Deno / Workers / Edge
pnpm add @unsearch/sdk

import { UnSearch } from "@unsearch/sdk"

const client = new UnSearch({ apiKey: process.env.UNSEARCH_API_KEY! })
const hits = await client.search({ query: "EU AI Act Article 12 logging" })
const verify = await client.verifyClaim({
  claim: "EU AI Act full enforcement begins August 2026",
  source_url: hits.results[0].url,
})

Self-host on your own Cloudflare account

For ICP-2 (regulated companies) — your data stays in your perimeter, you control the signing keys, audit logs retain up to 10 years.

git clone https://github.com/Rakesh1002/unsearch.git
cd unsearch
cp .env.example .env   # fill in CLOUDFLARE_ACCOUNT_ID + CLOUDFLARE_API_TOKEN
wrangler login
wrangler containers deploy   # deploys FastAPI + SearXNG to CF Containers (GA April 2026)
wrangler deploy              # deploys Hono edge + MCP + dashboard

Full self-host guide at [docs/quickstart.md](./docs/quickstart.md) and [docs/deployment/](./docs/deployment/).

For a Docker-Compose stack instead (no Cloudflare account required):

docker compose up -d
# minimal stack (no Celery/Postgres if you only need search + verify):
docker compose -f docker-compose.quickstart.yml up -d

---

What's in the repo

unsearch/
├── backend/                # FastAPI backend — search, extract, verify, audit (Python 3.11+)
│   ├── app/                #   Python module (`from app.X import Y`)
│   ├── alembic/            #   Postgres migrations
│   ├── tests/              #   pytest suite
│   ├── Dockerfile          #   Self-host image
│   └── Dockerfile.cloudflare #   CF Containers image
├── apps/                   # TypeScript / Python packages (pnpm workspace)
│   ├── workers/            #   Cloudflare Workers edge — Hono router, MCP transport, Durable Objects, D1 schema, containers.toml
│   ├── web/                #   Next.js dashboard on Cloudflare Workers (@opennextjs/cloudflare)
│   ├── sdk-ts/             #   @unsearch/sdk — TypeScript SDK
│   ├── sdk-py/             #   unsearch — Python SDK (sync + async)
│   ├── sdk-llamaindex/     #   @unsearch/llamaindex — LlamaIndex retriever
│   └── mcp-server/         #   @unsearch/mcp-server — MCP server (stdio + hosted /mcp)
├── infra/                  # Operational config (self-host stack + CF Container sidecars)
│   ├── nginx/              #   Reverse-proxy for self-host TLS
│   ├── monitoring/         #   Prometheus + Grafana provisioning
│   └── searxng/            #   SearXNG meta-search engine config (production settings.yml)
├── docs/                   # Architecture, API reference, strategy, ADRs, runbooks
├── scripts/                # Setup + ops scripts (manage.sh, setup-stripe.sh, …)
└── docker-compose*.yml     # Self-host stacks (build context = root; mount paths from infra/)

The architecture (Workers fronting FastAPI on Cloudflare Containers GA, with D1 / KV / Vectorize / R2 / Queues / Durable Objects + SearXNG sidecar) is documented in [docs/cloudflare-architecture.md](./docs/cloudflare-architecture.md). The five new ADRs that drove the 2026-05-28 reposition are at [docs/adr/](./docs/adr/README.md) (#0009 through #0013). ADR-0006 has a 2026-05-28 amendment describing the directory restructure (app/ → backend/app/, ops into infra/).

---

Honest feature status

> See [docs/feature-matrix.md](./docs/feature-matrix.md) for the full table. TL;DR below.

Shipped (production, end-to-end tested)

• Web search (/api/v1/search, /api/v1/agent/search) — signed citation envelope per result
• Neural / semantic search + auto-prompt + highlights + similar — Exa-compatible
• Multi-engine aggregation via SearXNG (70+ engines, sidecar container)
• Scraping (static, JavaScript, PDF, multi-engine), extraction, deep crawl
• Signed citation envelope (HMAC v1, WACZ-aligned) + R2/local snapshot store
• Claim verification (/api/v1/verify/claim) and citation diff (/api/v1/verify/citation)
• Per-API-key audit log (/api/v1/audit)
• RAG with Cloudflare Vectorize + bge-m3 embeddings + 4-tier Workers AI model selector
• Stripe billing — checkout + portal + subscriptions
• Cloudflare edge — Workers, Workers AI, Vectorize, Queues, Durable Objects, D1, KV, R2
• TypeScript SDK (@unsearch/sdk), Python SDK (unsearch), LlamaIndex retriever (@unsearch/llamaindex)
• MCP server at api.unsearch.dev/mcp and npx @unsearch/mcp-server

In flight / upcoming

• ✏️ Cloudflare Containers deploy (FastAPI + SearXNG sidecar) — requires enabling container binding in wrangler.toml
• ✏️ Dashboard at app.unsearch.dev, edge at api.unsearch.dev — domain + domain activation
• ✏️ MCP registry submission + HN launch

In beta (code paths exist, hardening in flight)

• Knowledge graph — entity extraction, relationship mapping (/api/v1/knowledge/*)
• Topic monitoring + webhook alerts (/api/v1/monitor/topics)
• Source credibility (/api/v1/verify/source)
• Deep research agent (/api/v1/agent/research)
• Predictive search (/api/v1/neural/predictive)

On the roadmap

• Google OAuth wiring (keys in .env.example; flow not yet wired)
• LangChain + Vercel AI SDK adapters
• WACZ public-key signing (Month 7+, replacing HMAC v1)
• BYO storage (S3 / GCS / Azure Blob) for snapshot store on self-host
• SOC 2 Type II (Month 9), HIPAA BAA (Month 6), ISO 42001 (Month 9–12)
• BYOC beyond Cloudflare (AWS / GCP / Azure deploy templates) — Month 10+
• SAML / OIDC SSO + RBAC — Enterprise tier

---

Path forward

The full ordered roadmap lives in [docs/roadmap.md](./docs/roadmap.md). The 3-week rebuild plan lives at ~/.claude/plans/app-unsearch-dev-is-not-deployed-luminous-wilkinson.md. At a glance:

Week 1 — Deploy what exists + reposition

1. Container deploy of FastAPI + SearXNG sidecar on CF Containers GA
2. Frontend pnpm cf:build && pnpm cf:deploy to app.unsearch.dev
3. Hono edge to api.unsearch.dev
4. Strategy docs + landing copy rewrite (this PR)

Week 2 — Verification wedge

1. R2 citation snapshot store with sha256 content addressing
2. WACZ-aligned signed envelopes (HMAC v1)
3. verify/citation + verify/claim endpoints GA
4. Dashboard verify + audit views

Distribution + launch

1. ✅ Hono-hosted MCP server at /mcp exposing search, extract, research, verify_claim
2. ✅ npx @unsearch/mcp-server package
3. MCP registry + awesome-mcp-servers submission
4. HN launch + outreach to first 10 named ICP-1 customers

Month 2–6

• LangChain + Vercel AI SDK adapters
• Self-host quickstart + wrangler deploy UX polish
• WACZ export endpoint
• BYO storage on self-host
• SOC 2 Type I (Month 4) → Type II (Month 9)
• First self-host v2 customer with BAA + DPA + SSO

---

SDKs

| Language | Package | Repo | |----------|---------|------| | Python (sync + async) | unsearch | [apps/sdk-py](./apps/sdk-py/) | | TypeScript / Node / Edge | @unsearch/sdk | [apps/sdk-ts](./apps/sdk-ts/) | | LlamaIndex retriever | @unsearch/llamaindex | [apps/sdk-llamaindex](./apps/sdk-llamaindex/) | | MCP server | @unsearch/mcp-server | [apps/mcp-server/](./apps/mcp-server/) | | LangChain | community PR in flight | — |

All SDKs cover the same surface — search, extract, research, neural search, RAG (with streaming), ingest, verify, topic monitoring, plus a tavily_search / tavilySearch drop-in.

---

API reference

93 endpoints across 14 routers. Live OpenAPI docs at:

• Local: http://localhost:8000/docs
• Production: api.unsearch.dev/docs

Headline endpoints

| Endpoint | Description | |----------|-------------| | POST /api/v1/search | Native UnSearch search — returns signed citation envelope per result | | POST /api/v1/agent/search | Tavily-compatible drop-in (compatibility surface) | | POST /api/v1/agent/research | Deep research agent (Durable Object, multi-step, audit-trailed) | | POST /api/v1/agent/extract | Web extraction with envelope | | POST /api/v1/verify/citation | Snapshot pin + live diff | | POST /api/v1/verify/claim | Span-level claim grading via Workers AI | | POST /api/v1/verify/source | 🔶 Source credibility | | GET /api/v1/audit | Per-API-key audit log | | POST /api/v1/neural/search | Exa-compatible neural search | | POST /api/v1/rag/query | RAG over your Vectorize namespace (SSE streaming) | | GET /mcp | Streamable HTTP MCP server |

Full reference: [docs/API_REFERENCE.md](./docs/APIREFERENCE.md). Worked examples: [docs/API_EXAMPLES.md](./docs/APIEXAMPLES.md). Envelope schema: [docs/citation-envelope.md](./docs/citation-envelope.md).

---

AI pipeline

UnSearch routes generation, embeddings, reranking, and the verify_claim grader through Cloudflare Workers AI. Pick a tier per request.

| Tier | Model | Use case | |------|-------|----------| | fast | llama-3.1-8b-instruct-fast | Cheap, low-latency answers, simple queries | | balanced | llama-3.3-70b-instruct-fp8-fast | Default — general answers + verify_claim grader | | reasoning | qwq-32b | Multi-step reasoning, agent workflows | | production | gpt-oss-120b | Highest-quality enterprise traffic |

Embeddings: bge-m3 (1024 dims) into Cloudflare Vectorize.

Full pipeline: [docs/ai-pipeline.md](./docs/ai-pipeline.md).

---

Configuration

See [.env.example](./.env.example) for the canonical list. The two required values for the hosted Cloudflare features:

CLOUDFLARE_ACCOUNT_ID="..."
CLOUDFLARE_API_TOKEN="..."

Optional infra defaults assume the Docker Compose stack:

SEARXNG_URL="http://searxng:8080"
REDIS_URL="redis://localhost:6379"
DATABASE_URL="postgresql://unsearch:${POSTGRES_PASSWORD:-changeme}@localhost:5432/unsearch"

Stripe billing, SMTP, OAuth, monitoring — all documented in [docs/configuration/env-variables.md](./docs/configuration/env-variables.md).

---

Development

# Python backend (everything backend-related lives in backend/)
python -m venv venv && source venv/bin/activate
pip install -r backend/requirements.txt
cd backend && uvicorn app.main:app --reload --port 8000

# Frontend (Next.js on Workers)
pnpm --filter @unsearch/web dev

# Edge worker (Hono on CF Workers)
cd workers && pnpm dev

# Python SDK
cd apps/sdk-py && pip install -e ".[test]" && pytest -q

# TypeScript SDK
pnpm --filter @unsearch/sdk build && pnpm --filter @unsearch/sdk test

# Backend tests (from backend/ where pytest.ini lives)
cd backend && pytest tests/unit/ -v --cov=app
cd backend && pytest tests/integration/ -

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [Rakesh1002](https://github.com/Rakesh1002)
- **Source:** [Rakesh1002/unsearch](https://github.com/Rakesh1002/unsearch)
- **License:** Apache-2.0
- **Homepage:** https://app.unsearch.dev

Install and usage instructions live in the source repository linked above.