# Textrawl

> Turn your documents into agent memory. MCP server with hybrid semantic + keyword search over emails, PDFs, notes, and past conversations. Works with Claude, ChatGPT, Cursor, and any MCP client.

- **Type:** MCP server
- **Install:** `agentstack add mcp-jeffgreendesign-textrawl`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [jeffgreendesign](https://agentstack.voostack.com/s/jeffgreendesign)
- **Installs:** 0
- **Category:** [Content & Media](https://agentstack.voostack.com/c/content-and-media)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [jeffgreendesign](https://github.com/jeffgreendesign)
- **Source:** https://github.com/jeffgreendesign/textrawl
- **Website:** https://www.textrawl.com/

## Install

```sh
agentstack add mcp-jeffgreendesign-textrawl
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Textrawl

[](LICENSE)
[](https://nodejs.org)
[](https://www.typescriptlang.org)
[](https://www.postgresql.org)
[](https://neon.tech)
[](https://openai.com)
[](https://ollama.com)
[](https://modelcontextprotocol.io)

**Your second brain, wired for AI.**

Textrawl is a personal knowledge server with persistent memory, searchable documents, and proactive insights. Import your emails, PDFs, and notes — then ask questions, recall past conversations, and discover connections you missed. Access it all through MCP, the web dashboard, or the REST API.

## How It Works

```text
┌──────────────┐         ┌───────────────────────────────────────────┐
│              │         │           Your Second Brain               │
│  MCP Client  │◄───────►│                                           │
│  (Claude,    │   MCP   │   Documents    Memory     Conversations   │
│   ChatGPT)   │         │   ┌────────┐  ┌───────┐  ┌────────────┐  │
└──────────────┘         │   │ Emails │  │ Facts │  │Past sessions│  │
                         │   │ PDFs   │  │People │  │ Summaries  │  │
┌──────────────┐         │   │ Notes  │  │ Links │  │ Context    │  │
│   Dashboard  │◄───────►│   └───┬────┘  └──┬────┘  └─────┬──────┘  │
│  (Web UI)    │  REST   │       └──────────┼──────────────┘         │
└──────────────┘         │                  ▼                        │
                         │     ┌───────────────────────────┐         │
                         │     │  Hybrid Search + Fusion   │         │
                         │     └───────────────────────────┘         │
                         │                  │                        │
                         │                  ▼                        │
                         │      Insights · Daily Briefing            │
                         └───────────────────────────────────────────┘
                                            ▲
                                            │
                                 ┌──────────┴──────────┐
                                 │                     │
                            Desktop App            CLI Tools
                           (drag & drop)        (batch import)
```

## Why Textrawl?

**Beyond keyword search.** Most search tools only match exact words. Textrawl combines semantic understanding (finds "automobile" when you search "car") with traditional keyword matching — so you get relevant results without missing exact phrases.

**Your data, your choice.** Use OpenAI's embeddings for best accuracy, Google AI for multimodal support, or run locally with Ollama and local Postgres to keep document text and embeddings on your machine.

**Import everything.** Emails from Gmail exports, PDFs from your research, saved web pages, images, audio files, Google Takeout archives — Textrawl converts them into searchable knowledge where the relevant converter/provider is configured.

## Features

| Feature | Description |
|---------|-------------|
| **Hybrid Search** | Vector similarity + full-text search with Reciprocal Rank Fusion |
| **Persistent Memory** | Remember facts about people, projects, and concepts across sessions |
| **Conversation Recall** | Save and query past conversation context across sessions |
| **Proactive Insights** | Automatically discover connections, patterns, and outliers in your knowledge |
| **Daily Briefing** | Summary of recent additions, new insights, and resurfaced knowledge |
| **Unified RAG** | `ask` tool searches documents, memory, and conversations in one query |
| **Web Dashboard** | Command center with knowledge explorer, timeline, agent orchestration, and applets |
| **Multimodal** | Process images (Claude vision) and audio (Whisper transcription) alongside documents |
| **Desktop App** | Drag-and-drop file conversion and upload (macOS, Windows, Linux) |
| **Multi-Format** | PDF, DOCX, XLSX, PPTX, HTML, MBOX/EML emails, Google Takeout |
| **MCP + REST + WebSocket** | MCP tools, REST API, and real-time WebSocket events |
| **Agent Discovery** | A2A protocol at `/.well-known/agent.json` for agent-to-agent interaction |
| **Flexible Embeddings** | OpenAI, Google AI, or Ollama (free, local) |
| **Smart Chunking** | Paragraph-aware splitting with overlap for context |
| **CLI Tools** | Batch processing for large archives |
| **Cloud Ready** | Deploy to Docker, Cloud Run, or any container platform; large uploads require GCS/Cloud Tasks configuration |

## Privacy Model

Textrawl is self-hosted, but data leaves your machine when you configure cloud services. Document text, chunks, embeddings, extracted memories, conversation summaries, images, or audio may be sent to providers such as OpenAI/Google embeddings, Anthropic/OpenAI/Google extraction, Neon/Supabase/RDS, Cloud Run, or GCS. For sensitive data, prefer Ollama/local Postgres and disable cloud LLM extraction/insights.

## Quick Start

### 1. Set Up the Server

```bash
git clone https://github.com/jeffgreendesign/textrawl.git
cd textrawl
pnpm install
pnpm setup    # Interactive setup for credentials
pnpm dev      # Start the server
```

### 2. Set Up Your Database

1. Create a free project at [neon.tech](https://neon.tech)
2. Copy the **pooled connection string** from the Neon dashboard into `DATABASE_URL` in `.env`
3. Run the base schema against your database:

   ```bash
   psql $DATABASE_URL -f scripts/setup-db.sql
   ```

   (Use `setup-db-ollama.sql` for Ollama / `setup-db-ollama-v2.sql` for `nomic-embed-text-v2-moe` / `setup-db-google.sql` for Google AI)
4. (Optional) For memory tools: `psql $DATABASE_URL -f scripts/setup-db-memory.sql`
5. (Optional) For conversation tools: `psql $DATABASE_URL -f scripts/setup-db-conversation.sql`
6. (Optional) For Row Level Security hardening: `psql $DATABASE_URL -f scripts/security-rls.sql`

### 3. Connect Claude Desktop

Add to your Claude config (`~/Library/Application Support/Claude/claude_desktop_config.json`). Create this file if it doesn't exist:

```json
{
  "mcpServers": {
    "textrawl": {
      "command": "npx",
      "args": [
        "mcp-remote",
        "http://localhost:3000/mcp",
        "--header",
        "Accept: application/json, text/event-stream"
      ]
    }
  }
}
```

**Note:** Requires Node.js 22+. If using nvm, ensure your default is set: `nvm alias default 22`

If you've set `API_BEARER_TOKEN` in `.env`, add the auth header:

```json
"--header",
"Authorization: Bearer "
```

Restart Claude Desktop - you'll now see Textrawl's tools available.

### 3b. Connect ChatGPT Desktop (Alternative)

ChatGPT Desktop supports MCP servers natively (Pro/Plus required):

1. Open **Settings → Connectors → Advanced → Developer mode**
2. Add a new connector with your server URL: `http://localhost:3000/mcp`
3. If using auth, add the `Authorization: Bearer ` header

See [OpenAI MCP documentation](https://platform.openai.com/docs/mcp) for details.

### 4. Add Your Documents

Imported documents, extracted memories, and conversation summaries are stored in your configured database/storage until deleted. Treat a Textrawl server as single-tenant unless you have added your own user isolation. Set `API_BEARER_TOKEN`, restrict CORS with `ALLOWED_ORIGINS`, and avoid importing third-party or private data without consent. Use `forget_entity` and `delete_conversation` to remove memory/conversation data, and `list_documents`/`update_document` to audit imported documents.

**Option A: Desktop App** (easiest)

```bash
pnpm desktop:dev
```

Drag files onto the window to convert and upload.

**Option B: CLI** (for batch imports)

```bash
pnpm convert -- mbox ~/Mail/archive.mbox
pnpm upload -- ./converted/
```

## Documentation

| Guide | Description |
|-------|-------------|
| [Database Sizing](docs/guides/database-requirements.mdx) | Vector dimensions, index counts, and storage estimates by embedding provider |
| [CLI Tools](docs/cli/) | Batch conversion and upload from command line |
| [Security](docs/guides/security-hardening.mdx) | Row Level Security and access controls |

---

## Configuration

| Variable | Required | Description |
|----------|----------|-------------|
| `DATABASE_URL` | Yes | Neon (or any PostgreSQL) pooled connection string |
| `DATABASE_URL_UNPOOLED` | No | Direct connection for schema migrations (optional) |
| `EMBEDDING_PROVIDER` | No | `openai` (default), `ollama`, or `google` |
| `OPENAI_API_KEY` | If OpenAI | For text-embedding-3-small (1536d) |
| `OLLAMA_BASE_URL` | If Ollama | Default: `http://localhost:11434` |
| `OLLAMA_MODEL` | If Ollama | Default: `nomic-embed-text` |
| `GOOGLE_AI_API_KEY` | If Google | For gemini-embedding-2-preview (3072d) |
| `GOOGLE_EMBEDDING_MODEL` | If Google | Default: `gemini-embedding-2-preview` |
| `API_BEARER_TOKEN` | Prod only | Min 32 chars (`openssl rand -hex 32`) |
| `PORT` | No | Default: 3000 |
| `LOG_LEVEL` | No | debug, info, warn, error |
| `ALLOWED_ORIGINS` | No | Comma-separated CORS origins |
| `ENABLE_MEMORY` | No | Enable memory tools (default: true); requires `setup-db-memory.sql` |
| `ENABLE_CONVERSATIONS` | No | Enable conversation memory tools (default: true); requires `setup-db-conversation.sql` |
| `ENABLE_INSIGHTS` | No | Enable proactive insight tools (default: true) |
| `ENABLE_MEMORY_EXTRACTION` | No | Enable LLM-based memory extraction (default: false) |
| `ANTHROPIC_API_KEY` | If extraction | Required for `extract_memories` tool |
| `EXTRACTION_MODEL` | No | Model for extraction (default: claude-haiku-4-5-20251001) |
| `INSIGHT_MODEL` | No | Model for insight synthesis (default: claude-sonnet-4-6) |
| `COMPACT_RESPONSES` | No | Token-efficient responses (default: true) |
| `CHUNKING_MODE` | No | `fixed` (default) or `semantic` (embedding-based splits) |
| `SEMANTIC_SIMILARITY_THRESHOLD` | No | Semantic split sensitivity 0–1 (default: 0.5) |
| `REDIS_URL` | No | Redis URL for shared rate limiting across instances |
| `GOOGLE_CLIENT_ID` | OAuth | OAuth 2.0 client ID (all four OAuth vars required together) |
| `GOOGLE_CLIENT_SECRET` | OAuth | OAuth 2.0 client secret |
| `OAUTH_JWT_SECRET` | OAuth | Min 32-char secret for JWT signing |
| `OAUTH_ALLOWED_EMAILS` | OAuth | Comma-separated email allowlist (optional) |
| `OAUTH_SERVER_URL` | OAuth | Public server URL for OAuth redirect |
| `PG_REPORT_DIR` | No | pg_analyze report directory (default: ./reports/pg-analysis) |

## Tool surfaces (compact vs full)

Textrawl advertises one of three tool surfaces via the `MCP_TOOLSET` environment
variable. The default **`normal`** surface is a small set of workflow tools — the
recommended interface for personal/family assistants. This follows the current MCP
maintainer guidance (Anthropic, _Writing effective tools for AI agents_): **fewer
tools, consolidated by workflow into distinct, typed, well-named tools** — not a
single "intent" dispatcher (which would discard per-tool schemas, annotations, and
the name signals models use to pick tools). The MCP spec defines no standard
tool-filtering primitive, so `MCP_TOOLSET` is a server-local convention; host/harness
**lazy tool loading** remains the canonical way to surface fewer tools at a time.

| `MCP_TOOLSET` | Tools advertised |
|---|---|
| `normal` (**default**) | Workflow tools: `ask`, `search`, `get_document`, `capture`, `daily_briefing`, `timeline` (+ `remember` when `ENABLE_MEMORY=true`). Set `EXPOSE_ADMIN_TOOLS=true` to also expose read-only diagnostics (`health_check`, `get_stats`, insight + Postgres tools). |
| `full` | Workflow tools + diagnostics + all original granular tools (backward compatible). |
| `legacy` | Exactly the original tool set (no workflow tools). |

**Recommended exposure:** personal/family bots → `MCP_TOOLSET=normal` and
`EXPOSE_ADMIN_TOOLS=false` (so destructive `forget_entity`/`delete_conversation` are
never reachable); admin/dev contexts → `MCP_TOOLSET=full`.

**Model guidance (GPT-5.5, Claude Sonnet, Claude Haiku):** express intent through the
workflow tool that matches the job — `ask` to answer a question across sources,
`search` for raw matches, `capture` to save a note or URL, `remember` to store facts,
`timeline`/`daily_briefing` for temporal/briefing views. Use `audience` to scope who
an answer is for; `family_shared`/`public_safe` automatically exclude private memory,
conversations, and insights.

## MCP Tools

Read-only tools (`search`, `get_document`, `list_documents`, `query_memory`, `query_conversations`, `get_stats`, `health_check`) include `outputSchema` and return `structuredContent` for programmatic consumption alongside the text `content` response.

### Workflow Tools (compact `normal` surface)

| Tool | Description |
|------|-------------|
| `capture` | Save content to the knowledge base — `kind="note"` (title+content) or `kind="url"` (fetch + clip). Consolidates `add_note` + `save_url`. |
| `remember` | Write structured knowledge to the memory graph — `facts` and/or `relations`. Consolidates `remember_fact` + `build_knowledge` + `relate_entities`. |

`ask`, `search`, `get_document`, `daily_briefing`, and `timeline` (documented below)
complete the workflow surface. `remember` is only advertised when `ENABLE_MEMORY=true`
(omitted otherwise). `ask`/`search` accept `audience` + `allowCrossProfile` for
privacy scoping.

### Document Tools

| Tool | Description |
|------|-------------|
| `search` | Hybrid semantic + full-text search. Set `includeMemories`/`includeConversations` for cross-source fusion. `audience` scopes private sources. |
| `get_document` | Retrieve document by ID |
| `list_documents` | List with pagination and filtering |
| `update_document` | Update title and/or tags |
| `add_note` | Add markdown note to knowledge base |

### Memory Tools (Persistent Memory)

Enable with `ENABLE_MEMORY=true` (default). Requires `scripts/setup-db-memory.sql` or `setup-db-memory-ollama.sql`.

| Tool | Description |
|------|-------------|
| `remember_fact` | Store facts about entities (people, projects, concepts) |
| `build_knowledge` | Store multiple facts and relations in a single batch call |
| `query_memory` | Query the memory graph (`mode: 'search' \| 'entity' \| 'list'`) |
| `relate_entities` | Create relationships between entities |
| `forget_entity` | Delete an entity and all its memories |
| `extract_memories` | Extract entities and facts from text using LLM |

### Conversation Tools (Conversation Memory)

Enable with `ENABLE_CONVERSATIONS=true` (default). Requires running one of the conversation schema scripts:

- `scripts/setup-db-conversation.sql` (OpenAI embeddings, 1536d)
- `scripts/setup-db-conversation-ollama.sql` (Ollama v1 - nomic-embed-text, 1024d)
- `scripts/setup-db-conversation-ollama-v2.sql` (Ollama v2 - nomic-embed-text-v2-moe, 768d)
- `scripts/setup-db-conversation-google.sql` (Google AI - gemini-embedding-2-preview, 3072d)

| Tool | Description |
|------|-------------|
| `save_conversation_context` | Save conversation summary and turns for recall |
| `query_conversations` | Query past conversations (`mode: 'search' \| 'get' \| 'list'`) |
| `delete_conversation` | Delete a conversation session |

### Insight Tools (Proactive Discovery)

Enable with `ENABLE_INSIGHTS=true` (default).

| Tool | Description |
|------|-------------|
| `get_insights` | View discovered cross-source connections and patterns |
| `discover_connections` | Trigger an insight scan across the knowledge base |
| `dismiss_insight` | Dismiss an insight from the queue |

### Stats

| Tool | Description |
|------|-------------|
| `get_stats` | Statistics across all features (`scope: 'all' \| 'knowledge' \| 'memory' \| 'conversations' \| 'insights'`) |
| `health_check` | Check the health of all subsystems — returns pass/fail per component with overall status (`verbose` for latency and counts) |

### Unified Tools

| Tool | Description |
|------|-------------|
| `ask` | Unified RAG search across all knowledge sources (documents, memory, conversations, insights) |
| `daily_briefing` | Generate a daily briefing with recent additions, new insights, and resurfaced

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [jeffgreendesign](https://github.com/jeffgreendesign)
- **Source:** [jeffgreendesign/textrawl](https://github.com/jeffgreendesign/textrawl)
- **License:** MIT
- **Homepage:** https://www.textrawl.com/

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** yes
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-jeffgreendesign-textrawl
- Seller: https://agentstack.voostack.com/s/jeffgreendesign
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.