Web Scraper
Extract content from any web page. Provides a tiered approach from lightweight fetch to full browser rendering. Use when web_fetch fails or pages require JavaScript.
Install
$ agentstack add skill-yesterday-ai-skills-web-scraper ✓ scanned · ✓ verified, works with Claude Code, Cursor, and more.
Security review
✓ PassedNo issues found. Passed automated security review. · v0.1.0 How review works →
- ✓ Prompt-injection patterns
- ✓ Secret / credential exfiltration
- ✓ Dangerous shell & filesystem operations
- ✓ Untrusted network calls
- ✓ Known-malicious package signatures
What it can access
- ✓ Network access No
- ✓ Filesystem access No
- ✓ Shell / process execution No
- ✓ Environment & secrets No
- ✓ Dynamic code execution No
From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.
Verified badge
Passed review? Show it. Paste this badge into your README, it links to the public security report.
Links to your public security report.[](https://agentstack.voostack.com/security/report/skill-yesterday-ai-skills-web-scraper) Reliability & compatibility
Declared compatibility
Compatibility is declared by the source manifest. End-to-end runtime verification is coming, see below.
We're building live execution health for every listing: tool-call success rate, median latency, uptime, and last-checked timestamps, measured, not self-reported. It isn't live yet, so we don't show numbers we can't stand behind.
How agent discovery & health will work →About
Web Scraper
Extract readable content from any URL using the best available method.
Tiered Approach (use in order)
Tier 1: web_fetch (Built-in, fast, no deps)
Best for: Static pages, APIs, markdown docs, RSS feeds.
web_fetch url="https://example.com" extractMode="markdown"Strengths: Zero setup, fast (~200ms), low cost. Weakness: No JavaScript rendering. Fails on SPAs, login walls, Cloudflare.
Tier 2: web_search (Built-in, Brave API)
Best for: Finding URLs, getting snippets, quick facts.
web_search query="topic keyword"Strengths: Fast, structured results with snippets. Weakness: Needs Brave API key. Rate limited (2000/month free).
Tier 3: Playwright Headless (Full rendering)
Best for: JavaScript-heavy pages, SPAs, login walls, social media.
node skills/web-scraper/scripts/scrape.js "https://example.com"Strengths: Renders everything a real browser would. Handles Cloudflare, JS, dynamic content. Weakness: Slow (~5-8s), needs Chromium installed, heavier on resources.
Tier 4: DuckDuckGo Lite (Fallback search)
Best for: When Brave API is unavailable.
web_fetch url="https://lite.duckduckgo.com/lite?q=search+terms"Strengths: No API key needed. Weakness: Gets blocked after ~10 queries. Captcha.
Decision Tree
Need content from URL?
├── Static page / API / docs → Tier 1 (web_fetch)
├── Need to find URLs first → Tier 2 (web_search)
├── Page needs JS / is SPA / social media → Tier 3 (Playwright)
└── No Brave API key → Tier 4 (DuckDuckGo)Setup (Tier 3 only)
npm install playwright
npx playwright install chromium
npx playwright install-deps chromium # System libs, may need sudoSee Also
- [x-reader](../x-reader/SKILL.md) -- Specialized for Twitter/X content
Source & license
This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.
- Author: Yesterday-AI
- Source: Yesterday-AI/skills
- License: MIT
- Homepage: https://github.com/Yesterday-AI/skills
Install and usage instructions live in the source repository linked above.
Reviews
No reviews yet, be the first.
Write a review
Versions
- v0.1.0 Imported from the upstream source.