Localvectordb

LocalVectorDB

A high-performance, document-first vector database with SQLite + FAISS backend, featuring intelligent chunking, unified search, and optional HTTP server.

[](https://pypi.org/project/localvectordb/) [](https://pypi.org/project/localvectordb/) [](LICENSE) [](https://github.com/thomas-villani/localvectordb/actions/workflows/test.yml) [](https://thomas-villani.github.io/localvectordb/) [](https://github.com/astral-sh/ruff) [](https://github.com/astral-sh/uv)

> Beyond basic RAG. LocalVectorDB pairs a zero-infrastructure SQLite + FAISS core with capabilities most vector stores don't have: > - 🧬 Hierarchical retrieval — search a three-level document → section → chunk hierarchy, so you can match a whole section (not just a stray sentence) in long, structured documents. > - ✅ Reverse-RAG fact-checking — ground LLM-generated text against your corpus, flagging unsupported or contradicted claims with citations. > - 📊 Document comparison & visualization — synteny ribbons and chord diagrams that show how two documents (or a document's own chunks) relate.

Contents

• [Quick Start](#-quick-start) — install, index, search
• [Use with Claude Code & other AI agents](#-use-with-claude-code--other-ai-agents)
• [Features](#-features)
• [Server Deployment](#️-server-deployment)
• [TypeScript SDK](#-typescript-sdk)
• [API Reference](#-api-reference)
• [CLI Reference](#️-cli-reference)
• [Architecture](#️-architecture)
• [File Extraction](#-file-extraction)
• [Configuration Options](#️-configuration-options)
• [Production Deployment](#-production-deployment)
• [Examples](#-examples)
• [Contributing](#-contributing)

📖 Full documentation:

🚀 Quick Start

Installation

LocalVectorDB is a standard PyPI package — install it with uv (recommended) or pip.

# Add to your project with uv (recommended)
uv add localvectordb

# For server features (optional)
uv add "localvectordb[server]"

# For all file extraction formats (optional)
uv add "localvectordb[server,file-extraction]"

Prefer pip? Every command above has a pip equivalent, e.g. pip install "localvectordb[server,file-extraction]".

To run the CLI/server without adding it to a project, use uv's tool runner:

uvx --from "localvectordb[server]" lvdb serve

Basic Usage

from localvectordb import VectorDB

# Create or connect to a database
db = VectorDB("my_docs", "./data")

# Add documents
doc_ids = db.upsert([
    "Python is a programming language",
    "Machine learning with neural networks"
])

# Search documents
results = db.query("programming", k=5)
for result in results:
    print(f"{result.id}: {result.content} (score: {result.score:.3f})")

# Get specific document
doc = db.get(doc_ids[0])
print(f"Content: {doc.content}")

With Metadata Schema

from localvectordb import VectorDB
from localvectordb.core import MetadataField, MetadataFieldType

# Define metadata schema
schema = {
    'title': MetadataField(type=MetadataFieldType.TEXT, indexed=True),
    'author': MetadataField(type=MetadataFieldType.TEXT, indexed=True),
    'created_date': MetadataField(type=MetadataFieldType.DATE, indexed=True),
    'tags': MetadataField(type=MetadataFieldType.JSON)
}

db = VectorDB("articles", "./data", metadata_schema=schema)

# Add documents with metadata
db.upsert(
    documents=["Article about Python programming"],
    metadata=[{
        'title': 'Python Guide',
        'author': 'Jane Doe',
        'created_date': '2024-01-15',
        'tags': ['python', 'programming', 'tutorial']
    }]
)

# Search with metadata filters
results = db.query(
    "programming tutorial",
    filters={'author': 'Jane Doe', 'tags': {'$contains': 'python'}}
)

Remote Server Usage

from localvectordb import VectorDB

# Use HTTP server (automatically detected by URL)
db = VectorDB(
    "my_docs", 
    "http://localhost:8000",
    api_key="your_api_key"
)

# Same API as local database
doc_ids = db.upsert(["Remote document content"])
results = db.query("content", search_type="hybrid")

🤖 Use with Claude Code & other AI agents

LocalVectorDB ships a built-in Model Context Protocol server, so an AI agent can search your knowledge bases directly — no glue code. It is read-only by default, which makes it safe to point at a corpus you care about.

uv add "localvectordb[mcp]"     # or: pip install "localvectordb[mcp]"

Build a knowledge base with the CLI first (rich formats like PDF and DOCX are extracted to Markdown automatically):

lvdb create technical_docs --embedding-model nomic-embed-text
lvdb db technical_docs add ./docs/*.md ./manual.pdf

Claude Code

Register the server once, from your project directory:

claude mcp add lvdb   -e LVDB_MCP_MODE=read-only   -e LVDB_MCP_DATABASES_ROOT=/path/to/databases   -- lvdb mcp serve

Everything after -- is the launch command, passed through untouched. Use -s project to write a committed .mcp.json your team shares, -s user to make it available in every project; the default scope is local (this project, just you). Then claude mcp list to check status, /mcp inside a session to manage it, and claude mcp remove lvdb to undo.

Claude Desktop, and any other MCP client

Add the same server to claude_desktop_config.json (or your client's equivalent config — the mcpServers block is a shared convention):

{
  "mcpServers": {
    "lvdb": {
      "type": "stdio",
      "command": "lvdb",
      "args": ["mcp", "serve"],
      "env": {
        "LVDB_MCP_MODE": "read-only",
        "LVDB_MCP_DATABASES_ROOT": "/path/to/databases"
      }
    }
  }
}

What the agent can do

The server exposes focused tools — query_database, find_related_documents, filter_documents, get_document (a whole document, or a chunk / line range / section of one), and list_databases. Tool sets are configurable per deployment, so you can expose only what a given agent needs.

Granting writes is explicit — lvdb mcp serve --mode read-write (or LVDB_MCP_MODE=read-write) additionally enables upsert_documents, create_database, and friends. Only do this for databases the agent is meant to modify.

See the MCP documentation for the full tool list, configuration reference, and security guidance.

✨ Features

🗃️ Document-First Architecture

• Smart Chunking: Position-tracking chunking — the default chunker reconstructs documents byte-for-byte from their chunks
• Metadata Schema: Structured, indexed metadata fields with validation
• Unified API: Single interface for vector, keyword, and hybrid search
• In-place Patch API: Edit a stored document with exact find/replace (or span splice) instead of re-sending the whole content — with an optional expect_hash precondition to guard against lost updates. Available in the library, HTTP API, MCP tool, CLI, and JS SDK

🔍 Advanced Search

• Vector Search: Semantic similarity via pluggable embedding providers — Ollama, OpenAI, Google, Jina, HuggingFace (Inference API + local), and Sentence Transformers
• Keyword Search: Full-text search with SQLite FTS5
• Hybrid Search: Combined vector + keyword with configurable weighting
• Reranking: Optional cross-encoder reranking via Jina, Sentence Transformers, or HuggingFace
• Metadata Filtering: MongoDB-style queries on structured metadata
• Document Scoring: 11 chunk-to-document aggregation strategies for tuning relevance

🧬 Hierarchical Retrieval

• Three-Level Hierarchy: Index and search at document, section, and chunk granularity
• Automatic Section Detection: Sections derived from document structure (Markdown headings by default, custom patterns supported)
• No Extra Embedding Cost: Section/document vectors are centroids of existing chunk embeddings
• Section Metadata: Pluggable extractors (heading path, keywords, word/char counts, or your own)

✅ Reverse-RAG Fact-Checking

• Grounding Verification: Check LLM-generated text against your databases claim-by-claim
• Citations & Contradictions: Each claim scored, cited to a source excerpt, and flagged if contradicted
• Multi-Provider LLMs: Works with Anthropic, OpenAI, or Gemini clients (auto-detected)

📊 Document Comparison & Visualization

• Similarity & Neighbors: Compare documents, find nearest neighbors, build similarity matrices
• Embedding Maps: t-SNE / PCA projections with clustering
• Synteny & Chord Diagrams: Visualize chunk-level alignment between documents or within one

🌐 Flexible Deployment

• Local Database: Direct SQLite + FAISS for maximum performance
• HTTP Server: RESTful API with permission-based authentication, rate limiting, CORS
• Remote Client: Seamless local/remote switching via factory pattern
• Multi-Worker: Redis-based coordination for distributed deployments

📄 File Processing

• Text Extraction: PDF, DOCX, PPTX, XLSX, RTF, EPUB support
• Batch Upload: Multi-file processing with metadata extraction
• Format Detection: Automatic MIME type detection and processing

🤖 AI / LLM Integration

• MCP Server: Built-in Model Context Protocol server for Claude Desktop, Claude Code, and other MCP clients
• Read-Only by Default: Safe knowledge-base access; opt into read-write explicitly
• TypeScript SDK: First-class browser/Node client (@localvectordb/sdk)

🛠️ Developer Experience

• CLI Tools: Database management, server control, interactive shell
• Configuration: TOML/JSON config with environment variable support
• Comprehensive Logging: Structured logging with performance monitoring
• Type Safety: Full type annotations and validation

🖥️ Server Deployment

Start the Server

# Quick start: with no config file, serves on localhost:8000 with built-in
# defaults. Run `lvdb config init` first to customize (host, auth, CORS, ...).
lvdb serve

# Production configuration
lvdb --config ./config.toml serve --host 0.0.0.0 --port 8000

Configuration

Create a configuration file:

# Interactive setup wizard
lvdb config init --interactive

# Production setup with Redis
lvdb config init --redis-registry redis://localhost:6379/1 \
                  --enable-cache --cache-type redis \
                  --enable-rate-limiting --enable-cors \
                  --enable-auth

Example configuration (config.toml):

[database]
root_dir = "./databases"
chunk_size = 500
chunking_method = "sentences"
chunk_overlap = 1

[embedding]
provider = "ollama"
model = "nomic-embed-text"

[server]
host = "0.0.0.0"
port = 8000
enable_rate_limiting = true
rate_limit = "100 per minute"

# Authentication and CORS live under the [server.security] table.
[server.security]
require_api_key = true
cors_enabled = true
cors_allowed_origins = ["http://localhost:3000"]

API Key Management

# Create API key with permission level
lvdb auth create-key --description "Production API" --permission-level read_write

# Create read-only key for analytics
lvdb auth create-key --description "Analytics Dashboard" --permission-level read_only

# List keys with their permissions
lvdb auth list-keys --active-only

# Revoke key
lvdb auth revoke-key key_20241201_abc123

🟦 TypeScript SDK

A zero-dependency TypeScript/JavaScript client is available for the HTTP server (Node.js 18+ and modern browsers):

npm install @localvectordb/sdk

import { LocalVectorDBClient } from "@localvectordb/sdk";

const client = new LocalVectorDBClient({ baseUrl: "http://localhost:8000" });
const db = client.database("my_docs");

await db.upsert(["First document", "Second document"]);
const results = await db.query("search text", { search_type: "hybrid", k: 5 });

See [sdk/js/README.md](sdk/js/README.md) for the full SDK API.

📚 API Reference

Core Methods

upsert(documents, metadata=None, ids=None)

Insert or update documents.

# Single document
db.upsert("Document content")

# Multiple documents with metadata
doc_ids = db.upsert(
    documents=["Doc 1", "Doc 2"],
    metadata=[{"type": "article"}, {"type": "blog"}],
    ids=["doc_1", "doc_2"]
)

query(query, search_type='hybrid', k=10, filters=None)

Unified search interface.

# Vector search
results = db.query("search text", search_type="vector", k=5)

# Hybrid search with metadata filter
results = db.query(
    "machine learning",
    search_type="hybrid",
    vector_weight=0.5,
    filters={"category": "AI"}
)

# Keyword search
results = db.query("exact phrase", search_type="keyword")

> Note: Filter fields (and metadata keys on upsert) must be declared in the > database's metadata_schema. Filtering on an undeclared field raises > DatabaseError; undeclared metadata keys are dropped on upsert with a warning.

get(ids) / delete(ids) / exists(ids)

Document management.

# Single document
doc = db.get("doc_1")
exists = db.exists("doc_1")
deleted_count = db.delete("doc_1")

# Multiple documents
docs = db.get(["doc_1", "doc_2"])
exist_flags = db.exists(["doc_1", "doc_2"])
deleted_count = db.delete(["doc_1", "doc_2"])

patch(doc_id, ops, *, expect_hash=None, metadata=None)

Edit a document in place without re-sending its whole content. Ops resolve against the current content, touch disjoint spans, and apply atomically.

# Exact find/replace (must match exactly `count` times, default 1)
result = db.patch("doc_1", [{"op": "replace", "find": "draft", "replace": "final"}])
print(result.updated, result.new_hash, result.ops_applied)

# Span splice + append/prepend
db.patch("doc_1", [{"op": "splice", "start": 0, "end": 5, "text": "Hello"}])
db.patch("doc_1", [{"op": "append", "text": " (revised)"}])

# Optimistic concurrency — raises PatchConflictError if the doc changed
doc = db.get("doc_1")
db.patch("doc_1", [{"op": "replace", "find": "v1", "replace": "v2"}],
         expect_hash=doc.content_hash)

filter(where=None, order_by=None, limit=None, offset=0)

MongoDB-style filtering on metadata.

# Simple filters
docs = db.filter(where={"author": "Jane Doe", "status": "published"})

# Complex queries with operators
docs = db.filter(
    where={"created_date": {"$gte": "2024-01-01"}},
    order_by="created_date DESC",
    limit=10
)

# Logical operators and pattern matching
docs = db.filter(
    where={"$and": [{"author": {"$like": "%Smith%"}}, {"rating": {"$gt": 4.0}}]}
)

HTTP API Endpoints

| Method | Endpoint | Description | |--------|----------|-------------| | GET | /api/v1/databases | List databases | | POST | /api/v1/databases | Create database | | GET | /api/v1/databases/{db}/info | Database info | | POST | /api/v1/databases/{db}/documents | Upsert documents | | GET | /api/v1/databases/{db}/documents/{id} | Get document | | PATCH | /api/v1/databases/{db}/documents/{id} | Update document | | DELETE | /api/v1/databases/{db}/documents/{id} | Delete document | | POST | /api/v1/databases/{db}/query | Search documents | | POST | /api/v1/databases/{db}/query/stream | Stream results (SSE) | | POST | /api/v1/databases/{db}/filter | Filter documents | | POST | /api/v1/databases/{db}/upload | Upload files | | POST | /api/v1/databases/{db}/compare | Compare documents | | POST | /api/v1/databases/{db}/nearest-neighbors | Find similar documents |

Example API usage:

# Create database
curl -X POST http://localhost:8000/api/v1/databases \
  -H "Authorization: Bearer your_api_key" \
  -H "Content-Type: application/json" \
  -d '{"name": "my_db"}'

# Search documents
curl -X POST http://localhost:8000/api/v1/databases/my_db/query \

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [thomas-villani](https://github.com/thomas-villani)
- **Source:** [thomas-villani/localvectordb](https://github.com/thomas-villani/localvectordb)
- **License:** MIT
- **Homepage:** https://thomas-villani.github.io/localvectordb/

Install and usage instructions live in the source repository linked above.