# Scrape Web

> >

- **Type:** Skill
- **Install:** `agentstack add skill-omnigentx-jarvis-scrape-web`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [omnigentx](https://agentstack.voostack.com/s/omnigentx)
- **Installs:** 0
- **Category:** [Web & Browser](https://agentstack.voostack.com/c/web-and-browser)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [omnigentx](https://github.com/omnigentx)
- **Source:** https://github.com/omnigentx/jarvis/tree/main/backend/.fast-agent/skills/scrape-web

## Install

```sh
agentstack add skill-omnigentx-jarvis-scrape-web
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Web Content Retrieval

Escalation chain — follow this order strictly:
1. serpapi (search/get) → Fast, stable, for most search queries. Always use `gl=vn`, `hl=vi`.
2. ScraplingServer get → When accessing a specific URL directly
3. ScraplingServer fetch → If get returns 403/empty
4. ScraplingServer stealthy_fetch → If fetch is still blocked

DO NOT use ScraplingServer for search queries. Only use it when you need direct URL access that serpapi cannot provide.

### When to Use What

| Scenario | Tool |
|----------|------|
| General information search | `serpapi` → search |
| Google/Bing results | `serpapi` |
| Access specific user-provided URL | ScraplingServer `get` |
| URL returns 403/blocked | ScraplingServer `fetch` |
| Cloudflare/anti-bot protection | ScraplingServer `stealthy_fetch` |
| JS-rendered SPA | ScraplingServer `fetch` + `network_idle: true` |

---

## ScraplingServer — Fallback Details

### Per-Site Recommendations

| Site | Tool |
|------|------|
| LinkedIn, Glassdoor, GitHub, Reddit | `get` |
| Telegram public channel | `get` + URL: `https://t.me/s/CHANNEL` |
| Twitter/X, Twitch, Facebook | `fetch` + `network_idle: true` |
| TikTok profile/page | `stealthy_fetch` + `network_idle: true`, `wait: 4000` |
| TikTok comments | `get` via internal API (see below) |
| Cloudflare-protected | `stealthy_fetch` + `solve_cloudflare: true` |

### Key Parameters

- `extraction_type`: `"text"` · `"markdown"` · `"html"`
- `main_content_only: true` — strips nav/footer noise
- `css_selector` — target specific elements
- `network_idle: true` + `wait: 2000` — for lazy-loaded JS content
- `disable_resources: true` — faster (fetch/stealthy_fetch only)

### Bulk Fetching

```
tool: bulk_get / bulk_fetch
urls: ["https://...", "https://..."]
```

---

## TikTok Comments API

Comments don't render in DOM — call the internal API directly (no login needed):

```
tool: get
url: https://www.tiktok.com/api/comment/list/
params: { aweme_id: "VIDEO_ID", count: "10", cursor: "0", aid: "1988" }
headers: { Referer: "https://www.tiktok.com/@USER/video/VIDEO_ID" }
extraction_type: "html"
main_content_only: false
```

Response JSON fields: `comments[].text`, `comments[].user.nickname`, `comments[].digg_count`, `total`, `has_more`. Paginate by incrementing `cursor` by 10.

To get VIDEO_ID: `stealthy_fetch` the profile page as `html`, extract with regex `/@[\w.]+/video/(\d+)`.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [omnigentx](https://github.com/omnigentx)
- **Source:** [omnigentx/jarvis](https://github.com/omnigentx/jarvis)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-omnigentx-jarvis-scrape-web
- Seller: https://agentstack.voostack.com/s/omnigentx
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.