Deep Research Mcp

Deep Research MCP Server

The Deep Research MCP Server is a Model Context Protocol (MCP) compliant server designed to perform comprehensive web research. It leverages Tavily's powerful Search and new Crawl APIs to gather extensive, up-to-date information on a given topic. The server then aggregates this data along with documentation generation instructions into a structured JSON output, perfectly tailored for Large Language Models (LLMs) to create detailed and high-quality markdown documents.

Features

• Multi-Step Research: Combines Tavily's AI-powered web search with deep content crawling for thorough information gathering.
• Structured JSON Output: Provides well-organized data (original query, search summary, detailed findings per source, and documentation instructions) optimized for LLM consumption.
• Configurable Documentation Prompt: Includes a comprehensive default prompt for generating high-quality technical documentation. This prompt can be:
• Overridden by setting the DOCUMENTATION_PROMPT environment variable.
• Further overridden by passing a documentation_prompt argument directly to the tool.
• Configurable Output Path: Specify where research documents and images should be saved through:
• Environment variable configuration
• JSON configuration
• Direct parameter in tool calls
• Granular Control: Offers a wide range of parameters to fine-tune both the search and crawl processes.
• MCP Compliant: Designed to integrate seamlessly into MCP-based AI agent ecosystems.

Prerequisites

• Node.js (version 18.x or later recommended)
• npm (comes with Node.js) or Yarn

Installation

Installing via Smithery

To install deep-research-mcp for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @pinkpixel/dev-deep-research-mcp --client claude

Option 1: Using with NPX (Recommended for quick use)

You can run the server directly using npx without a global installation:

npx @pinkpixel/deep-research-mcp

Option 2: Global Installation (Optional)

npm install -g @pinkpixel/deep-research-mcp

Then you can run it using:

deep-research-mcp

Option 3: Local Project Integration or Development

1. Clone the repository (if you want to modify or contribute):

``bash git clone [https://github.com/your-username/deep-research-mcp.git](https://github.com/your-username/deep-research-mcp.git) cd deep-research-mcp ``

1. Install dependencies:

``bash npm install ``

Configuration

The server requires a Tavily API key and can optionally accept a custom documentation prompt.

{
  "mcpServers": {
    "deep-research": {
      "command": "npx",
      "args": [
        "-y",
        "@pinkpixel/deep-research-mcp"
      ],
      "env": {
        "TAVILY_API_KEY": "tvly-YOUR_ACTUAL_API_KEY_HERE", // Required
        "DOCUMENTATION_PROMPT": "Your custom, detailed instructions for the LLM on how to generate markdown documents from the research data...", // Optional - if not provided, the default prompt will be used
        "SEARCH_TIMEOUT": "120", // Optional - timeout in seconds for search requests (default: 60)
        "CRAWL_TIMEOUT": "300", // Optional - timeout in seconds for crawl requests (default: 180)
        "MAX_SEARCH_RESULTS": "10", // Optional - maximum search results to retrieve (default: 7)
        "CRAWL_MAX_DEPTH": "2", // Optional - maximum crawl depth (default: 1)
        "CRAWL_LIMIT": "15", // Optional - maximum URLs to crawl per source (default: 10)
        "FILE_WRITE_ENABLED": "true", // Optional - enable file writing capability (default: false)
        "ALLOWED_WRITE_PATHS": "/home/user/research,/home/user/documents", // Optional - comma-separated allowed directories (default: user home directory)
        "FILE_WRITE_LINE_LIMIT": "300" // Optional - maximum lines per file write operation (default: 200)
      }
    }
  }
}

1\. Tavily API Key (Required)

Set the TAVILY_API_KEY environment variable to your Tavily API key.

Methods:

• .env file: Create a .env file in the project root (if running locally for development):

``env TAVILY_API_KEY="tvly-YOUR_ACTUAL_API_KEY" ``

• Directly in command line:

``bash TAVILY_API_KEY="tvly-YOUR_ACTUAL_API_KEY" npx @pinkpixel/deep-research-mcp ``

• System Environment Variable: Set it in your operating system's environment variables.

2\. Custom Documentation Prompt (Optional)

You can override the default comprehensive documentation prompt by setting the DOCUMENTATION_PROMPT environment variable.

Methods (in order of precedence):

1. Tool Argument: The documentation_prompt parameter passed when calling the deep-research-tool takes highest precedence
2. Environment Variable: If no parameter is provided in the tool call, the system checks for a DOCUMENTATION_PROMPT environment variable
3. Default Value: If neither of the above are set, the comprehensive built-in default prompt is used

Setting via .env file:

DOCUMENTATION_PROMPT="Your custom, detailed instructions for the LLM on how to generate markdown..."

Or directly in command line:

DOCUMENTATION_PROMPT="Your custom prompt..." TAVILY_API_KEY="tvly-YOUR_KEY" npx @pinkpixel/deep-research-mcp

3\. Output Path Configuration (Optional)

You can specify where research documents and images should be saved. If not configured, a default path in the user's Documents folder with a timestamp will be used.

Methods (in order of precedence):

1. Tool Argument: The output_path parameter passed when calling the deep-research-tool takes highest precedence
2. Environment Variable: If no parameter is provided in the tool call, the system checks for a RESEARCH_OUTPUT_PATH environment variable
3. Default Path: If neither of the above are set, a timestamped subfolder in the user's Documents folder is used: ~/Documents/research/YYYY-MM-DDTHH-MM-SS/

Setting via .env file:

RESEARCH_OUTPUT_PATH="/path/to/your/research/folder"

Or directly in command line:

RESEARCH_OUTPUT_PATH="/path/to/your/research/folder" TAVILY_API_KEY="tvly-YOUR_KEY" npx @pinkpixel/deep-research-mcp

4\. Timeout and Performance Configuration (Optional)

You can configure timeout and performance settings via environment variables to optimize the tool for your specific use case or deployment environment:

Available Environment Variables:

• SEARCH_TIMEOUT - Timeout in seconds for Tavily search requests (default: 60)
• CRAWL_TIMEOUT - Timeout in seconds for Tavily crawl requests (default: 180)
• MAX_SEARCH_RESULTS - Maximum number of search results to retrieve (default: 7)
• CRAWL_MAX_DEPTH - Maximum crawl depth from base URL (default: 1)
• CRAWL_LIMIT - Maximum number of URLs to crawl per source (default: 10)

Setting via .env file:

SEARCH_TIMEOUT=120
CRAWL_TIMEOUT=300
MAX_SEARCH_RESULTS=10
CRAWL_MAX_DEPTH=2
CRAWL_LIMIT=15

Or directly in command line:

SEARCH_TIMEOUT=120 CRAWL_TIMEOUT=300 TAVILY_API_KEY="tvly-YOUR_KEY" npx @pinkpixel/deep-research-mcp

When to adjust these settings:

• Increase timeouts if you're experiencing timeout errors in LibreChat or other MCP clients
• Decrease timeouts for faster responses when working with simpler queries
• Increase limits for more comprehensive research (but expect longer processing times)
• Decrease limits for faster processing with lighter resource usage

5\. File Writing Configuration (Optional)

The server includes a secure file writing tool that allows LLMs to save research findings directly to files. This feature is disabled by default for security reasons.

Security Features:

• File writing must be explicitly enabled via FILE_WRITE_ENABLED=true
• Directory restrictions via ALLOWED_WRITE_PATHS (defaults to user home directory)
• Line limits per write operation to prevent abuse
• Path validation and sanitization
• Automatic directory creation

Configuration:

FILE_WRITE_ENABLED=true
ALLOWED_WRITE_PATHS=/home/user/research,/home/user/documents,/tmp/research
FILE_WRITE_LINE_LIMIT=500

Usage Example: Once enabled, LLMs can use the write-research-file tool to save content:

{
  "tool": "write-research-file",
  "arguments": {
    "file_path": "/home/user/research/quantum-computing-report.md",
    "content": "# Quantum Computing Research Report\n\n...",
    "mode": "rewrite"
  }
}

Security Considerations:

• Only enable file writing in trusted environments
• Use specific directory restrictions rather than allowing system-wide access
• Monitor file operations through server logs
• Consider using read-only directories for sensitive systems

Running the Server

• Development (with auto-reload):

If you've cloned the repository and are in the project directory:

``bash npm run dev ``

This uses nodemon and ts-node to watch for changes and restart the server.

• Production/Standalone:

First, build the TypeScript code:

``bash npm run build ``

Then, start the server:

``bash npm start ``

• With NPX or Global Install:

(Ensure environment variables are set as described in Configuration)

``bash npx @pinkpixel/deep-research-mcp ``

or if globally installed:

``bash deep-research-mcp ``

The server will listen for MCP requests on stdio.

How It Works

1. An LLM or AI agent makes a CallToolRequest to this MCP server, specifying the deep-research-tool and providing a query and other optional parameters.
2. The deep-research-tool first performs a Tavily Search to find relevant web sources.
3. It then uses Tavily Crawl to extract detailed content from each of these sources.
4. All gathered information (search snippets, crawled content, image URLs) is aggregated.
5. The chosen documentation prompt (default, ENV, or tool argument) is included.
6. The server returns a single JSON string containing all this structured data.
7. The calling LLM/agent uses this JSON output, guided by the documentation_instructions, to generate a comprehensive markdown document.

Using the deep-research-tool

This is the primary tool exposed by the server.

Output Structure

The tool returns a JSON string with the following structure:

{
  "documentation_instructions": "string", // The detailed prompt for the LLM to generate the markdown.
  "original_query": "string",         // The initial query provided to the tool.
  "search_summary": "string | null",  // An LLM-generated answer/summary from Tavily's search phase (if include_answer was true).
  "research_data": [                  // Array of findings, one element per source.
    {
      "search_rank": "number",
      "original_url": "string",           // URL of the source found by search.
      "title": "string",                  // Title of the web page.
      "initial_content_snippet": "string",// Content snippet from the initial search result.
      "search_score": "number | undefined",// Relevance score from Tavily search.
      "published_date": "string | undefined",// Publication date (if 'news' topic and available).
      "crawled_data": [                 // Array of pages crawled starting from original_url.
        {
          "url": "string",                // URL of the specific page crawled.
          "raw_content": "string | null", // Rich, extracted content from this page.
          "images": ["string", "..."]     // Array of image URLs found on this page.
        }
      ],
      "crawl_errors": ["string", "..."]   // Array of error messages if crawling this source failed or had issues.
    }
    // ... more sources
  ],
  "output_path": "string"             // Path where research documents and images should be saved.
}

Input Parameters

The deep-research-tool accepts the following parameters in its arguments object:

General Parameters

• query (string, required): The main research topic or question.
• documentation_prompt (string, optional): Custom prompt for LLM documentation generation.
• Description: If provided, this prompt will be used by the LLM. It overrides both the DOCUMENTATION_PROMPT environment variable and the server's built-in default prompt. If not provided here, the server checks the environment variable, then falls back to the default.
• output_path (string, optional): Path where generated research documents and images should be saved.
• Description: If provided, this path will be used for saving research outputs. It overrides the RESEARCH_OUTPUT_PATH environment variable. If neither is set, a timestamped folder in the user's Documents directory will be used.

Search Parameters (for Tavily Search API)

• search_depth (string, optional, default: "advanced"): Depth of the initial Tavily search.
• Options: "basic", "advanced". Advanced search is tailored for more relevant sources.
• topic (string, optional, default: "general"): Category for the Tavily search.
• Options: "general", "news".
• days (number, optional): For topic: "news", the number of days back from the current date to include search results.
• time_range (string, optional): Time range for search results (e.g., "d" for day, "w" for week, "m" for month, "y" for year).
• max_search_results (number, optional, default: 7): Maximum number of search results to retrieve and consider for crawling (1-20).
• chunks_per_source (number, optional, default: 3): For search_depth: "advanced", the number of content chunks to retrieve from each source (1-3).
• include_search_images (boolean, optional, default: false): Include a list of query-related image URLs from the initial search.
• include_search_image_descriptions (boolean, optional, default: false): Include image descriptions along with URLs from the initial search.
• include_answer (boolean or string, optional, default: false): Include an LLM-generated answer from Tavily based on search results.
• Options: true (implies "basic"), false, "basic", "advanced".
• include_raw_content_search (boolean, optional, default: false): Include the cleaned and parsed HTML content of each initial search result.
• include_domains_search (array of strings, optional, default: []): A list of domains to specifically include in the search results.
• exclude_domains_search (array of strings, optional, default: []): A list of domains to specifically exclude from the search results.
• search_timeout (number, optional, default: 60): Timeout in seconds for Tavily search requests.

Crawl Parameters (for Tavily Crawl API - applied to each URL from search)

• crawl_max_depth (number, optional, default: 1): Max depth of the crawl from the base URL. 0 means only the base URL, 1 means the base URL and links found on it, etc.
• crawl_max_breadth (number, optional, default: 5): Max number of links to follow per level of the crawl tree (i.e., per page).
• crawl_limit (number, optional, default: 10): Total number of links the crawler will process starting from a single root URL before stopping.
• crawl_instructions (string, optional): Natural language instructions for the crawler for how to approach crawling the site.
• crawl_select_paths (array of strings, optional, default: []): Regex patterns to select only URLs with specific path patterns for crawling (e.g., "/docs/.*").
• crawl_select_domains (array of strings, optional, default: []): Regex patterns to restrict crawling to specific domains or subdomains (e.g., "^docs\\.example\\.com$"). If crawl_allow_external is false (default) and this is empty, crawling is focused on the domain of the URL

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: pinkpixel-dev
• Source: pinkpixel-dev/deep-research-mcp
• License: MIT
• Homepage: https://pinkpixel.dev

Install and usage instructions live in the source repository linked above.