Anakin Web Data

Anakin: fetching web data

Pick the right tool first

Anakin has twenty-one tools across five groups. Choosing wrong is the most common and most expensive mistake — check this table before calling anything.

| The task | Tool | Skill | |---|---|---| | Read one known URL | scrape | this one | | Find what pages exist on a site | map | this one | | Collect many pages from one site | crawl | this one | | Find pages across the web for a query | search | anakin-research | | Answer a question needing many sources | agentic_search | anakin-research | | See what AI engines say about something | ai_visibility_search | anakin-research | | Extract data from a specific popular site | wire_discover → wire_read_action | anakin-wire | | Submit a form, add to cart, post content | wire_discover → wire_write_action | anakin-wire | | Watch a page for changes over time | monitor_create | anakin-monitoring | | Multi-step interaction no Wire action covers | browser_task | anakin-browser | | Reach login-protected content | session_list → pass sessionId | anakin-browser |

Check Wire before reaching for scrape or crawl on a well-known site. Amazon, Walmart, LinkedIn, Airbnb, Zillow and hundreds more have vetted, pre-built extractors that return clean structured data in one call. Scraping those sites by hand is slower, costs more, and gives you markdown you then have to parse. See the anakin-wire skill.

Escalate in cost order. scrape → scrape with useBrowser → a Wire action → browser_task. Each step is slower and more expensive than the last; do not start at the end.

scrape — one URL to markdown

Default behavior returns clean markdown. That is the right default; do not add options reflexively.

• generateJson: true — also run AI extraction and return typed fields. Use for

product pages, listings, articles — anywhere you want structured values rather than prose. Returns the JSON as the primary output.

• useBrowser: true — render with a stealth headless browser. **Slower and more

expensive.** Only for SPAs and JS-rendered content. Try the default first; if the markdown comes back empty or is missing the content you can see in a browser, retry with useBrowser: true.

• country — two-letter proxy egress code, defaults to "us". Set it when a

page is geo-restricted or price/inventory varies by market ("de", "in", …).

• forceFresh: true — skip the cache. Results cache for roughly 24h, which is

usually what you want. Only force fresh for genuinely live data (stock levels, prices you are about to act on).

• sessionId / sessionName — a saved browser session for login-protected

pages. Pair with useBrowser: true.

map — discover a site's URLs

Returns internal links, external links, and counts. Use it to understand structure before crawling, so the crawl is scoped instead of blind.

• limit (default 100, max 1000) — total URLs returned.
• depth (default 2, max 5) — link-hops followed.
• limitPerLevel (default 100) — breadth cap per level.
• search — keyword filter on path/title. Cheap way to find the section you

care about without crawling everything.

• includeSubdomains, includeExternalLinks — both default false.

crawl — bulk markdown from a site

For catalog ingestion or building a RAG corpus. Returns an array of pages, each with markdown and its own status — check per-page status, since a crawl can partially succeed.

• maxPages (default 10, max 500) — a hard cap, and the main cost lever.
• depth (default 1, max 5).
• includePatterns / excludePatterns — glob/regex URL filters. Use these.

An unscoped crawl burns credits on nav pages, tag archives, and pagination.

Working pattern

For anything beyond a single page, map first, then crawl the subset:

1. map the domain with a search filter to see what exists.
2. Derive includePatterns from the URLs that matter.
3. crawl with those patterns and a deliberate maxPages.

This is consistently cheaper and cleaner than crawling from the homepage and filtering afterward.

Cost and failure notes

• useBrowser is the biggest cost multiplier — leave it off unless the page

needs it.

• Every tool returns an error envelope rather than throwing. On failure, read

the message: it names the tool and the cause.

• API key comes from ANAKIN_API_KEY. Get one at https://anakin.io/dashboard —

free tier is 300 credits, no card.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: Anakin-Inc
• Source: Anakin-Inc/agent-skills
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.