Fetchaller Mcp

fetchaller-mcp

Fetch any website in Claude Code without permission prompts. Built-in web search, Reddit support, and automatic bot challenge bypass.

Why fetchaller?

Claude Code's built-in WebFetch asks permission for every new domain and blocks Reddit entirely. fetchaller fixes both:

• fetch: Read any URL — automatically bypasses Cloudflare, Akamai, and other bot challenges
• search: Web search via Google + DuckDuckGo
• browse_reddit: Browse subreddit listings (hot/new/top/rising)
• search_reddit: Search Reddit posts globally or within a subreddit
• search_marketplace: Search Kijiji, Craigslist, and Facebook Marketplace simultaneously with human-readable params (city name, category, price range)
• search_realtor: Search Canadian homes on realtor.ca for sale or rent with full filters (location, price, beds, baths, property/building type, ownership)
• get_aliexpress_product: AliExpress product details (price, specs, ratings, reviews)
• search_aliexpress: Search AliExpress products with price filters and sorting
• get_alibaba_product: Alibaba.com B2B product details (tiered pricing, MOQ, lead times, supplier info)
• search_alibaba: Search Alibaba.com B2B products

Quick Start

Local Installation (stdio mode)

# Clone and install
git clone https://github.com/Averyy/fetchaller-mcp.git
cd fetchaller-mcp
uv sync && patchright install chromium

# Add to Claude Code
claude mcp add fetchaller -- $(pwd)/.venv/bin/python -m fetchaller.main

Add permissions to ~/.claude/settings.json:

{
  "permissions": {
    "allow": [
      "mcp__fetchaller__fetch",
      "mcp__fetchaller__search",
      "mcp__fetchaller__browse_reddit",
      "mcp__fetchaller__search_reddit",
      "mcp__fetchaller__search_marketplace",
      "mcp__fetchaller__search_realtor",
      "mcp__fetchaller__get_aliexpress_product",
      "mcp__fetchaller__search_aliexpress",
      "mcp__fetchaller__get_alibaba_product",
      "mcp__fetchaller__search_alibaba"
    ]
  }
}

Restart Claude Code.

Recommended CLAUDE.md Addition

Add this to your project's CLAUDE.md (or global ~/.claude/CLAUDE.md) to instruct Claude to prefer fetchaller:

## Web Fetching & Search

**ALWAYS use fetchaller tools instead of WebFetch and WebSearch.** fetchaller has no domain restrictions and produces cleaner output.

- `mcp__fetchaller__fetch(url, maxTokens?, timeout?)` — Fetch any URL → clean markdown
- `mcp__fetchaller__search(query, page?)` — Web search (Google + DuckDuckGo)
- `mcp__fetchaller__browse_reddit(subreddit, sort?, time?, limit?)` — Browse subreddit listings
- `mcp__fetchaller__search_reddit(query, subreddit?, sort?, time?, limit?)` — Search Reddit posts
- `mcp__fetchaller__search_marketplace(query, location, platforms?, category?, sort?, condition?, min_price?, max_price?)` — Search Kijiji + Craigslist + Facebook Marketplace
- `mcp__fetchaller__search_realtor(location, transaction?, property_type?, building_type?, min_price?, max_price?, min_beds?, min_baths?, ownership?, sort?, page?)` — Search realtor.ca homes
- `mcp__fetchaller__get_aliexpress_product(product_id)` — AliExpress product details
- `mcp__fetchaller__search_aliexpress(query, page?, sort?, min_price?, max_price?)` — Search AliExpress
- `mcp__fetchaller__get_alibaba_product(product_id)` — Alibaba.com product details
- `mcp__fetchaller__search_alibaba(query, page?, sort?, min_price?, max_price?)` — Search Alibaba.com

Usage

The mcp__fetchaller__fetch tool is now available:

# Fetch a URL
fetch https://example.com

# Fetch with token limit
fetch https://example.com maxTokens=10000

# Fetch slow site with longer timeout
fetch https://slow-site.com maxTokens=25000 timeout=60

Web Search

# Search the web
search "python asyncio tutorial"

# Page 2 of results
search "python asyncio tutorial" page=2

Web Research Pattern

1. Use search to find URLs
2. Use fetch to read them

Tool Reference

fetch(url, maxTokens?, timeout?, raw?)

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | url | string | required | URL to fetch (http/https) | | maxTokens | number | 25000 | Max tokens to return | | timeout | number | 10 | Request timeout in seconds | | raw | boolean | false | Return raw HTML instead of markdown |

Returns

Clean markdown with:

• Page title as H1
• Scripts, styles, nav, footer, iframes removed
• HTML converted to markdown
• Redirects noted
• Content truncated at token limit

Edge Cases

| Scenario | Behavior | |----------|----------| | Invalid URL | Error message | | Non-200 response | Error + partial body | | JSON content | Returned as-is | | XML/RSS feeds | Returned as-is | | CSV files | Returned as-is | | Plain text | Returned as-is | | PDF files | Text extracted | | Timeout | Error after timeout (default 10s) | | Huge page | Truncated at maxTokens |

search(query, page?)

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | query | string | required | Search query | | page | number | 1 | Result page (1-indexed) |

Searches Google (primary) and DuckDuckGo (supplement) in parallel. Returns titles, URLs, and snippets. Page 2+ queries Google only.

Reddit Tools

Two tools for Reddit research:

browse_reddit - Browse Subreddit Listings

browse_reddit({
  subreddit: "LocalLLaMA",   // without r/ prefix
  sort: "hot",               // hot, new, top, rising
  time: "day",               // hour, day, week, month, year, all (for "top" only)
  limit: 10                  // 1-25
})

Returns post titles, scores, comment counts, and URLs. Use fetch to read full posts.

search_reddit - Search Posts

search_reddit({
  query: "best mass spectrometry software",
  subreddit: "labrats",      // optional - limit to subreddit
  sort: "relevance",         // relevance, hot, top, new, comments
  time: "year",              // hour, day, week, month, year, all
  limit: 10                  // 1-25
})

Returns matching posts with metadata. Use fetch to read full discussions.

URL Transformation

All Reddit URLs are automatically transformed to old.reddit.com for 65-70% token savings. Trailing slashes are added to avoid 301 redirects (~50-100ms latency savings):

| Input URL | Transformed To | |-----------|----------------| | www.reddit.com/r/foo | old.reddit.com/r/foo/ | | reddit.com/r/foo | old.reddit.com/r/foo/ | | old.reddit.com/r/foo | old.reddit.com/r/foo/ |

Rate Limits

Reddit allows ~10 unauthenticated API requests per minute. browse_reddit and search_reddit each use 1 API call. fetch uses HTML (no API call).

AliExpress & Alibaba Tools

get_aliexpress_product(product_id) - Product Details

Accepts a numeric product ID (e.g., 1005006027485365) or full URL. Returns price, specifications, ratings, and recent reviews via AliExpress's MTop API.

search_aliexpress(query, page?, sort?, min_price?, max_price?) - Search Products

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | query | string | required | Search query | | page | number | 1 | Page number (1-indexed) | | sort | string | "default" | default, orders, priceasc, pricedesc | | minprice | number | — | Minimum price filter | | maxprice | number | — | Maximum price filter |

get_alibaba_product(product_id) - B2B Product Details

Accepts a numeric product ID or full URL. Returns tiered pricing, MOQ, lead times, supplier info, and specifications.

search_alibaba(query, page?, sort?, min_price?, max_price?) - Search B2B Products

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | query | string | required | Search query | | page | number | 1 | Page number (1-indexed) | | sort | string | "default" | default, priceasc, pricedesc | | minprice | number | — | Minimum price filter (USD) | | maxprice | number | — | Maximum price filter (USD) |

Marketplace Search

search_marketplace — Search Kijiji, Craigslist, and Facebook Marketplace

Searches all three platforms concurrently with human-readable parameters and returns grouped results.

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | query | string | required | Search keywords | | location | string | required | City name (e.g. "toronto", "st catharines, ON", "seattle") | | platforms | string[] | all | Platforms to search: kijiji, craigslist, facebook | | category | string | "all" | all, cars, electronics, furniture, clothing, tools, free, bikes, phones, motorcycles, boats, rvs, autoparts, sporting, toys, baby | | sort | string | "date" | date, priceasc, pricedesc, relevance | | condition | string | — | new, likenew, good, fair | | minprice | number | — | Minimum price in dollars | | maxprice | number | — | Maximum price in dollars |

Kijiji is Canada-only and automatically skipped for US locations. Location matching supports exact names, common aliases (e.g. "niagara" → Hamilton CL area), and fuzzy matching for typos.

Real Estate Search

search_realtor — Search realtor.ca homes

Searches Canadian homes for sale or rent via realtor.ca's api2, with the full filter set so an assistant can narrow a home search. Returns listings with price, address, beds/baths, size, agent, and a realtor.ca URL.

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | location | string | required | City, neighbourhood, or postal code (e.g. "Ottawa", "Orleans, Ottawa", "M5V") | | transaction | string | "sale" | sale, rent | | propertytype | string | "any" | any, residential, condo, recreational, vacant-land, multi-family, agriculture, parking | | buildingtype | string | — | house, duplex, triplex, townhouse, apartment, other | | minprice | integer | — | Minimum price (sale) or monthly rent | | maxprice | integer | — | Maximum price (sale) or monthly rent | | minbeds | integer | — | Minimum bedrooms | | minbaths | integer | — | Minimum bathrooms | | ownership | string | — | freehold, condo | | sort | string | "newest" | newest, oldest, price-asc, price-desc | | page | integer | 1 | Result page (~20 per page, up to 600 returnable) |

Call fetch(url) on any listing URL for the full description, every property detail, and similar nearby homes. fetch also handles realtor.ca search/SEO/map pages (/{prov}/{city}/real-estate, /map) and all wellfound.com pages (startup job search, job detail, company profiles).

How It Works

1. Validates URL (http/https only)
2. Blocks private/internal IPs (SSRF protection with DNS rebinding prevention)
3. Fetches with browser-like TLS fingerprints via wafer (Rust/BoringSSL) — rotates Chrome versions automatically
4. If bot challenge detected: solves automatically (see Bot Challenge Bypass below)
5. Detects content type
6. For HTML: removes junk elements (nav, footer, ads, cookie banners), applies site-specific cleanup (25+ sites including GitHub, Reddit, HN, Wikipedia, Medium, Stack Overflow, Amazon, eBay, AliExpress, Alibaba, DigiKey, Mouser, realtor.ca, wellfound.com, plus Ashby/Greenhouse/Lever/Gem/Dayforce/Cornerstone/Workday/BambooHR/JazzHR/Work-at-a-Startup job boards with embed + white-label detection, and more), converts to markdown
7. For JSON/XML/CSV/text: returns raw
8. For PDF: extracts text
9. Truncates to token limit

Bot Challenge Bypass

fetchaller transparently bypasses bot challenges. First requests to protected sites take longer (total wall time = solve + fetch, typically 10-40s), but subsequent requests use cached cookies and are fast (~0.5s). The timeout parameter only controls the HTTP fetch — the browser solve has its own internal timeouts.

Supported Challenges

| Challenge | Method | Speed | |-----------|--------|-------| | Alibaba Cloud WAF (ACW) | Inline Python solver | ~1ms | | Alibaba Cloud WAF (TMD) | Inline warming + browser | ~5-10s | | Cloudflare Managed Challenge | Patchright browser solver | ~3-30s | | Akamai Bot Manager | Patchright browser solver | ~3-15s | | Amazon rate-limit/CAPTCHA | Patchright browser solver | ~3-10s | | DataDome, PerimeterX, Imperva | Patchright browser solver | ~3-10s | | Kasada | Browser CT token + Python SHA-256 PoW | ~3-10s | | GeeTest v4 slide CAPTCHA | CV notch detection + drag replay | ~5-15s | | reCAPTCHA v2 | Checkbox → audio (Whisper) → ONNX grid | ~5-30s |

All challenge solving is handled by wafer's BrowserSolver (Patchright-based). Cookies are cached per-domain so subsequent requests skip the challenge.

Requirements

Docker: Patchright's bundled Chromium is included in the image. The cookie-data volume persists solved cookies across restarts. No extra setup needed.

Local (stdio): Browser support (Patchright) is included by default. Run patchright install chromium after installing to download the browser binary. Docker includes it automatically.

Architecture

Content Processing

src/fetchaller/content/ handles HTML→markdown conversion. Each site module exports is_(url), SELECTORS_LIST, and optionally strip__junk(soup) / postprocess_(markdown):

• html.py — Generic pipeline + dispatch. Universal junk selectors, markdownify, whitespace cleanup. Generic JSON-LD Product fallback.
• amazon.py — All TLDs (.com, .ca, .co.uk, .de, etc.). CSS selectors, soup cleanup, regex post-processors.
• github.py — CSS selectors, URL transforms, file tree extraction, issue/PR/discussion extraction from embedded JSON.
• reddit.py — CSS selectors for old.reddit.com, URL transforms (www→old), post formatting.
• hackernews.py — CSS selectors, table unwrapping, story block reformatter.
• medium.py — CSS selectors (data-testid), HTML-based detection for unknown custom domains.
• huggingface.py — data-target attribute selectors, filter tag/button cleanup.
• stackoverflow.py — All Stack Exchange sites. CSS selectors, soup cleanup, regex post-processors.
• redflagdeals.py — RFD-specific CSS selectors, soup cleanup, regex post-processors.
• forums.py — Generic forum support (XenForo, vBulletin, phpBB, Discourse). RSS/Atom feed autodiscovery.
• wikipedia.py — CSS selectors for edit buttons, navboxes, TOC, reference lists.
• alibaba.py — Embedded JSON extraction (window.detailData, window.__page__data_sse10), soup cleanup.
• aliexpress.py — CSS selectors, soup cleanup, regex post-processors.
• craigslist.py — All city subdomains. CSS selectors, regex post-processors. Search URL detection for SAPI intercept.
• facebook_marketplace.py — URL detection only. GraphQL client in facebook_marketplace/ package.
• digikey.py — All TLDs. CSS selectors, soup cleanup. Behind Akamai (wafer handles). HTML fallback without API key.
• ebay.py — All TLDs. JSON-LD product extraction, search result DOM extraction (.s-item), regex post-processors.
• molex.py — JSON-LD Product extraction (additionalProperty specs). CSR site — specs only in structured data.
• mouser.py — All TLDs. CSS selectors, soup cleanup. Behind Akamai. HTML fallback without API key.
• soylent.py — Shopify store cleanup, inventory extraction from gsf_conversion_data.
• ti.py — Document viewer support for lazy-loaded datasheets.
• ashby.py / greenhouse.py / lever.py / gem.py / dayforce.py / cornerstone.py / workday.py / bamboohr.py / jazzhr.py / workatastartup.py — Job-board platforms. All preserve the source's own field names, enum values, and section titles. Each posting and (where supported) board listing is dispatched to the platform's API/JSON shell before the generic HTML pipeline. Five embed/white-label detectors run during the HTML phase so company career pages like synaptivemedical.com/job-openings (

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Averyy
• Source: Averyy/fetchaller-mcp
• License: MIT
• Homepage: https://www.fetchaller.com

Install and usage instructions live in the source repository linked above.