Llm Search

[](https://githubtocolab.com/snexus/llm-search/blob/main/notebooks/llmsearchgooglecolab_demo.ipynb)

pyLLMSearch - Advanced RAG

Documentation

The purpose of this package is to offer an advanced question-answering (RAG) system with a simple YAML-based configuration that enables interaction with a collection of local documents. Special attention is given to improvements in various components of the system in addition to basic LLM-based RAGs - better document parsing, hybrid search, HyDE, chat history, deep linking, re-ranking, the ability to customize embeddings, and more. The package is designed to work with custom Large Language Models (LLMs) – whether from OpenAI or installed locally.

Interaction with the package is supported through the built-in frontend, or by exposing an MCP server, allowing clients like Cursor, Windsurf or VSCode GH Copilot to interact with the RAG system.

Features

• Fast parsing and embedding of medium size document bases (tested on up to few gigabytes of markdown and pdfs)
• Incremental updates for new documents, without a need to re-index the entire document base.
• Supported document formats
• Build-in parsers:
• .md - Divides files based on logical components such as headings, subheadings, and code blocks. Supports additional features like cleaning image links, adding custom metadata, and more.
• .pdf - MuPDF-based parser.
• .docx - custom parser, supports nested tables.
• Other common formats are supported by Unstructured pre-processor:
• List of formats see here.
• FastAPI based API + MCP server, allowing communicating with RAG via any mcp client, including VSCode/Windsurf/Cursor and others.

• Deep linking into document sections - jump to an individual PDF page or a header in a markdown file.

• Allows interaction with embedded documents, internally supporting the following models and

methods (including locally hosted):

• OpenAI compatible models and APIs.
• HuggingFace models.

• Interoperability with LiteLLM + Ollama via OpenAI API, supporting hundreds of different models (see [Model configuration for LiteLLM](sample_templates/llm/litellm.yaml))

• SSE MCP Server enabling interface with popular MCP clients.

• Hybrid search and Reranking
• Dense embeddings from a folder of documents and stores them in a vector database (ChromaDB).
• The following embedding models are supported:
• Hugging Face embeddings.
• Sentence-transformers-based models.
• Instructor-based models.
• OpenAI embeddings.

• Sparse embeddings using SPLADE (https://github.com/naver/splade) to enable hybrid search (sparse + dense).

• Supports the "Retrieve and Re-rank" strategy for semantic search, see here.
• Besides the originally ms-marco-MiniLM cross-encoder, more modern bge-reranker-v2-m3 and zerank-2 is supported.

• Support for table parsing via open-source gmft (https://github.com/conjuncts/gmft) or Azure Document Intelligence.

• Optional support for image parsing using Gemini API.

• Supports HyDE (Hypothetical Document Embeddings) - see here.
• WARNING: Enabling HyDE (via config OR webapp) can significantly alter the quality of the results. Please make sure to read the paper before enabling.
• From my own experiments, enabling HyDE significantly boosts quality of the output on a topics where user can't formulate the quesiton using domain specific language of the topic - e.g. when learning new topics.

• Support for multi-querying, inspired by RAG Fusion - https://towardsdatascience.com/forget-rag-the-future-is-rag-fusion-1147298d8ad1
• When multi-querying is turned on (either config or webapp), the original query will be replaced by 3 variants of the same query, allowing to bridge the gap in the terminology and "offer different angles or perspectives" according to the article.

• Supprts optional chat history with question contextualization

• Other features
• Simple web interfaces.
• Ability to save responses to an offline database for future analysis.

Demo

Documentation

Browse Documentation

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: snexus
• Source: snexus/llm-search
• License: MIT

Install and usage instructions live in the source repository linked above.