# Localvectordb

> Document-first sqlite3+faiss vector database

- **Type:** MCP server
- **Install:** `agentstack add mcp-thomas-villani-localvectordb`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [thomas-villani](https://agentstack.voostack.com/s/thomas-villani)
- **Installs:** 0
- **Category:** [Databases](https://agentstack.voostack.com/c/databases)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [thomas-villani](https://github.com/thomas-villani)
- **Source:** https://github.com/thomas-villani/localvectordb
- **Website:** https://thomas-villani.github.io/localvectordb/

## Install

```sh
agentstack add mcp-thomas-villani-localvectordb
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# LocalVectorDB

A high-performance, document-first vector database with SQLite + FAISS backend, featuring intelligent chunking, unified search, and optional HTTP server.

[](https://pypi.org/project/localvectordb/)
[](https://pypi.org/project/localvectordb/)
[](LICENSE)
[](https://github.com/thomas-villani/localvectordb/actions/workflows/test.yml)
[](https://thomas-villani.github.io/localvectordb/)
[](https://github.com/astral-sh/ruff)
[](https://github.com/astral-sh/uv)

> **Beyond basic RAG.** LocalVectorDB pairs a zero-infrastructure SQLite + FAISS core with capabilities most vector stores don't have:
> - 🧬 **Hierarchical retrieval** — search a three-level *document → section → chunk* hierarchy, so you can match a whole section (not just a stray sentence) in long, structured documents.
> - ✅ **Reverse-RAG fact-checking** — ground LLM-generated text against your corpus, flagging unsupported or contradicted claims with citations.
> - 📊 **Document comparison & visualization** — synteny ribbons and chord diagrams that show how two documents (or a document's own chunks) relate.

## Contents

- [Quick Start](#-quick-start) — install, index, search
- [Use with Claude Code & other AI agents](#-use-with-claude-code--other-ai-agents)
- [Features](#-features)
- [Server Deployment](#️-server-deployment)
- [TypeScript SDK](#-typescript-sdk)
- [API Reference](#-api-reference)
- [CLI Reference](#️-cli-reference)
- [Architecture](#️-architecture)
- [File Extraction](#-file-extraction)
- [Configuration Options](#️-configuration-options)
- [Production Deployment](#-production-deployment)
- [Examples](#-examples)
- [Contributing](#-contributing)

📖 **Full documentation:** 

## 🚀 Quick Start

### Installation

LocalVectorDB is a standard PyPI package — install it with [uv](https://docs.astral.sh/uv/) (recommended) or pip.

```bash
# Add to your project with uv (recommended)
uv add localvectordb

# For server features (optional)
uv add "localvectordb[server]"

# For all file extraction formats (optional)
uv add "localvectordb[server,file-extraction]"
```

Prefer pip? Every command above has a pip equivalent, e.g. `pip install "localvectordb[server,file-extraction]"`.

To run the CLI/server without adding it to a project, use uv's tool runner:

```bash
uvx --from "localvectordb[server]" lvdb serve
```

### Basic Usage

```python
from localvectordb import VectorDB

# Create or connect to a database
db = VectorDB("my_docs", "./data")

# Add documents
doc_ids = db.upsert([
    "Python is a programming language",
    "Machine learning with neural networks"
])

# Search documents
results = db.query("programming", k=5)
for result in results:
    print(f"{result.id}: {result.content} (score: {result.score:.3f})")

# Get specific document
doc = db.get(doc_ids[0])
print(f"Content: {doc.content}")
```

### With Metadata Schema

```python
from localvectordb import VectorDB
from localvectordb.core import MetadataField, MetadataFieldType

# Define metadata schema
schema = {
    'title': MetadataField(type=MetadataFieldType.TEXT, indexed=True),
    'author': MetadataField(type=MetadataFieldType.TEXT, indexed=True),
    'created_date': MetadataField(type=MetadataFieldType.DATE, indexed=True),
    'tags': MetadataField(type=MetadataFieldType.JSON)
}

db = VectorDB("articles", "./data", metadata_schema=schema)

# Add documents with metadata
db.upsert(
    documents=["Article about Python programming"],
    metadata=[{
        'title': 'Python Guide',
        'author': 'Jane Doe',
        'created_date': '2024-01-15',
        'tags': ['python', 'programming', 'tutorial']
    }]
)

# Search with metadata filters
results = db.query(
    "programming tutorial",
    filters={'author': 'Jane Doe', 'tags': {'$contains': 'python'}}
)
```

### Remote Server Usage

```python
from localvectordb import VectorDB

# Use HTTP server (automatically detected by URL)
db = VectorDB(
    "my_docs", 
    "http://localhost:8000",
    api_key="your_api_key"
)

# Same API as local database
doc_ids = db.upsert(["Remote document content"])
results = db.query("content", search_type="hybrid")
```

## 🤖 Use with Claude Code & other AI agents

LocalVectorDB ships a built-in [Model Context Protocol](https://modelcontextprotocol.io/)
server, so an AI agent can search your knowledge bases directly — no glue code.
It is **read-only by default**, which makes it safe to point at a corpus you care
about.

```bash
uv add "localvectordb[mcp]"     # or: pip install "localvectordb[mcp]"
```

Build a knowledge base with the CLI first (rich formats like PDF and DOCX are
extracted to Markdown automatically):

```bash
lvdb create technical_docs --embedding-model nomic-embed-text
lvdb db technical_docs add ./docs/*.md ./manual.pdf
```

### Claude Code

Register the server once, from your project directory:

```bash
claude mcp add lvdb   -e LVDB_MCP_MODE=read-only   -e LVDB_MCP_DATABASES_ROOT=/path/to/databases   -- lvdb mcp serve
```

Everything after `--` is the launch command, passed through untouched. Use
`-s project` to write a committed `.mcp.json` your team shares, `-s user` to make
it available in every project; the default scope is `local` (this project, just
you). Then `claude mcp list` to check status, `/mcp` inside a session to manage
it, and `claude mcp remove lvdb` to undo.

### Claude Desktop, and any other MCP client

Add the same server to `claude_desktop_config.json` (or your client's equivalent
config — the `mcpServers` block is a shared convention):

```json
{
  "mcpServers": {
    "lvdb": {
      "type": "stdio",
      "command": "lvdb",
      "args": ["mcp", "serve"],
      "env": {
        "LVDB_MCP_MODE": "read-only",
        "LVDB_MCP_DATABASES_ROOT": "/path/to/databases"
      }
    }
  }
}
```

### What the agent can do

The server exposes focused tools — `query_database`, `find_related_documents`,
`filter_documents`, `get_document` (a whole document, or a chunk / line range /
section of one), and `list_databases`. Tool sets are configurable per deployment,
so you can expose only what a given agent needs.

Granting writes is explicit — `lvdb mcp serve --mode read-write` (or
`LVDB_MCP_MODE=read-write`) additionally enables `upsert_documents`,
`create_database`, and friends. Only do this for databases the agent is meant to
modify.

See the [MCP documentation](https://thomas-villani.github.io/localvectordb/mcp.html)
for the full tool list, configuration reference, and security guidance.

## ✨ Features

### 🗃️ **Document-First Architecture**
- **Smart Chunking**: Position-tracking chunking — the default chunker reconstructs documents byte-for-byte from their chunks
- **Metadata Schema**: Structured, indexed metadata fields with validation
- **Unified API**: Single interface for vector, keyword, and hybrid search
- **In-place Patch API**: Edit a stored document with exact find/replace (or span splice) instead of re-sending the whole content — with an optional `expect_hash` precondition to guard against lost updates. Available in the library, HTTP API, MCP tool, CLI, and JS SDK

### 🔍 **Advanced Search**
- **Vector Search**: Semantic similarity via pluggable embedding providers — Ollama, OpenAI, Google, Jina, HuggingFace (Inference API + local), and Sentence Transformers
- **Keyword Search**: Full-text search with SQLite FTS5
- **Hybrid Search**: Combined vector + keyword with configurable weighting
- **Reranking**: Optional cross-encoder reranking via Jina, Sentence Transformers, or HuggingFace
- **Metadata Filtering**: MongoDB-style queries on structured metadata
- **Document Scoring**: 11 chunk-to-document aggregation strategies for tuning relevance

### 🧬 **Hierarchical Retrieval**
- **Three-Level Hierarchy**: Index and search at *document*, *section*, and *chunk* granularity
- **Automatic Section Detection**: Sections derived from document structure (Markdown headings by default, custom patterns supported)
- **No Extra Embedding Cost**: Section/document vectors are centroids of existing chunk embeddings
- **Section Metadata**: Pluggable extractors (heading path, keywords, word/char counts, or your own)

### ✅ **Reverse-RAG Fact-Checking**
- **Grounding Verification**: Check LLM-generated text against your databases claim-by-claim
- **Citations & Contradictions**: Each claim scored, cited to a source excerpt, and flagged if contradicted
- **Multi-Provider LLMs**: Works with Anthropic, OpenAI, or Gemini clients (auto-detected)

### 📊 **Document Comparison & Visualization**
- **Similarity & Neighbors**: Compare documents, find nearest neighbors, build similarity matrices
- **Embedding Maps**: t-SNE / PCA projections with clustering
- **Synteny & Chord Diagrams**: Visualize chunk-level alignment between documents or within one

### 🌐 **Flexible Deployment**
- **Local Database**: Direct SQLite + FAISS for maximum performance
- **HTTP Server**: RESTful API with permission-based authentication, rate limiting, CORS
- **Remote Client**: Seamless local/remote switching via factory pattern
- **Multi-Worker**: Redis-based coordination for distributed deployments

### 📄 **File Processing**
- **Text Extraction**: PDF, DOCX, PPTX, XLSX, RTF, EPUB support
- **Batch Upload**: Multi-file processing with metadata extraction
- **Format Detection**: Automatic MIME type detection and processing

### 🤖 **AI / LLM Integration**
- **MCP Server**: Built-in [Model Context Protocol](https://modelcontextprotocol.io/) server for Claude Desktop, Claude Code, and other MCP clients
- **Read-Only by Default**: Safe knowledge-base access; opt into read-write explicitly
- **TypeScript SDK**: First-class browser/Node client (`@localvectordb/sdk`)

### 🛠️ **Developer Experience**
- **CLI Tools**: Database management, server control, interactive shell
- **Configuration**: TOML/JSON config with environment variable support
- **Comprehensive Logging**: Structured logging with performance monitoring
- **Type Safety**: Full type annotations and validation

## 🖥️ Server Deployment

### Start the Server

```bash
# Quick start: with no config file, serves on localhost:8000 with built-in
# defaults. Run `lvdb config init` first to customize (host, auth, CORS, ...).
lvdb serve

# Production configuration
lvdb --config ./config.toml serve --host 0.0.0.0 --port 8000
```

### Configuration

Create a configuration file:

```bash
# Interactive setup wizard
lvdb config init --interactive

# Production setup with Redis
lvdb config init --redis-registry redis://localhost:6379/1 \
                  --enable-cache --cache-type redis \
                  --enable-rate-limiting --enable-cors \
                  --enable-auth
```

Example configuration (`config.toml`):

```toml
[database]
root_dir = "./databases"
chunk_size = 500
chunking_method = "sentences"
chunk_overlap = 1

[embedding]
provider = "ollama"
model = "nomic-embed-text"

[server]
host = "0.0.0.0"
port = 8000
enable_rate_limiting = true
rate_limit = "100 per minute"

# Authentication and CORS live under the [server.security] table.
[server.security]
require_api_key = true
cors_enabled = true
cors_allowed_origins = ["http://localhost:3000"]
```

### API Key Management

```bash
# Create API key with permission level
lvdb auth create-key --description "Production API" --permission-level read_write

# Create read-only key for analytics
lvdb auth create-key --description "Analytics Dashboard" --permission-level read_only

# List keys with their permissions
lvdb auth list-keys --active-only

# Revoke key
lvdb auth revoke-key key_20241201_abc123
```

## 🟦 TypeScript SDK

A zero-dependency TypeScript/JavaScript client is available for the HTTP server
(Node.js 18+ and modern browsers):

```bash
npm install @localvectordb/sdk
```

```typescript
import { LocalVectorDBClient } from "@localvectordb/sdk";

const client = new LocalVectorDBClient({ baseUrl: "http://localhost:8000" });
const db = client.database("my_docs");

await db.upsert(["First document", "Second document"]);
const results = await db.query("search text", { search_type: "hybrid", k: 5 });
```

See [`sdk/js/README.md`](sdk/js/README.md) for the full SDK API.

## 📚 API Reference

### Core Methods

#### `upsert(documents, metadata=None, ids=None)`
Insert or update documents.

```python
# Single document
db.upsert("Document content")

# Multiple documents with metadata
doc_ids = db.upsert(
    documents=["Doc 1", "Doc 2"],
    metadata=[{"type": "article"}, {"type": "blog"}],
    ids=["doc_1", "doc_2"]
)
```

#### `query(query, search_type='hybrid', k=10, filters=None)`
Unified search interface.

```python
# Vector search
results = db.query("search text", search_type="vector", k=5)

# Hybrid search with metadata filter
results = db.query(
    "machine learning",
    search_type="hybrid",
    vector_weight=0.5,
    filters={"category": "AI"}
)

# Keyword search
results = db.query("exact phrase", search_type="keyword")
```

> **Note:** Filter fields (and metadata keys on upsert) must be declared in the
> database's `metadata_schema`. Filtering on an undeclared field raises
> `DatabaseError`; undeclared metadata keys are dropped on upsert with a warning.

#### `get(ids)` / `delete(ids)` / `exists(ids)`
Document management.

```python
# Single document
doc = db.get("doc_1")
exists = db.exists("doc_1")
deleted_count = db.delete("doc_1")

# Multiple documents
docs = db.get(["doc_1", "doc_2"])
exist_flags = db.exists(["doc_1", "doc_2"])
deleted_count = db.delete(["doc_1", "doc_2"])
```

#### `patch(doc_id, ops, *, expect_hash=None, metadata=None)`
Edit a document in place without re-sending its whole content. Ops resolve
against the current content, touch disjoint spans, and apply atomically.

```python
# Exact find/replace (must match exactly `count` times, default 1)
result = db.patch("doc_1", [{"op": "replace", "find": "draft", "replace": "final"}])
print(result.updated, result.new_hash, result.ops_applied)

# Span splice + append/prepend
db.patch("doc_1", [{"op": "splice", "start": 0, "end": 5, "text": "Hello"}])
db.patch("doc_1", [{"op": "append", "text": " (revised)"}])

# Optimistic concurrency — raises PatchConflictError if the doc changed
doc = db.get("doc_1")
db.patch("doc_1", [{"op": "replace", "find": "v1", "replace": "v2"}],
         expect_hash=doc.content_hash)
```

#### `filter(where=None, order_by=None, limit=None, offset=0)`
MongoDB-style filtering on metadata.

```python
# Simple filters
docs = db.filter(where={"author": "Jane Doe", "status": "published"})

# Complex queries with operators
docs = db.filter(
    where={"created_date": {"$gte": "2024-01-01"}},
    order_by="created_date DESC",
    limit=10
)

# Logical operators and pattern matching
docs = db.filter(
    where={"$and": [{"author": {"$like": "%Smith%"}}, {"rating": {"$gt": 4.0}}]}
)
```

### HTTP API Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/api/v1/databases` | List databases |
| `POST` | `/api/v1/databases` | Create database |
| `GET` | `/api/v1/databases/{db}/info` | Database info |
| `POST` | `/api/v1/databases/{db}/documents` | Upsert documents |
| `GET` | `/api/v1/databases/{db}/documents/{id}` | Get document |
| `PATCH` | `/api/v1/databases/{db}/documents/{id}` | Update document |
| `DELETE` | `/api/v1/databases/{db}/documents/{id}` | Delete document |
| `POST` | `/api/v1/databases/{db}/query` | Search documents |
| `POST` | `/api/v1/databases/{db}/query/stream` | Stream results (SSE) |
| `POST` | `/api/v1/databases/{db}/filter` | Filter documents |
| `POST` | `/api/v1/databases/{db}/upload` | Upload files |
| `POST` | `/api/v1/databases/{db}/compare` | Compare documents |
| `POST` | `/api/v1/databases/{db}/nearest-neighbors` | Find similar documents |

Example API usage:

```bash
# Create database
curl -X POST http://localhost:8000/api/v1/databases \
  -H "Authorization: Bearer your_api_key" \
  -H "Content-Type: application/json" \
  -d '{"name": "my_db"}'

# Search documents
curl -X POST http://localhost:8000/api/v1/databases/my_db/query \

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [thomas-villani](https://github.com/thomas-villani)
- **Source:** [thomas-villani/localvectordb](https://github.com/thomas-villani/localvectordb)
- **License:** MIT
- **Homepage:** https://thomas-villani.github.io/localvectordb/

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** yes
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-thomas-villani-localvectordb
- Seller: https://agentstack.voostack.com/s/thomas-villani
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.