# Download Gated Pdfs

> |

- **Type:** Skill
- **Install:** `agentstack add skill-kennethkhoocy-legal-scholarship-skills-download-gated-pdfs`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [kennethkhoocy](https://agentstack.voostack.com/s/kennethkhoocy)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [kennethkhoocy](https://github.com/kennethkhoocy)
- **Source:** https://github.com/kennethkhoocy/legal-scholarship-skills/tree/main/plugins/legal-scholarship/skills/download-gated-pdfs

## Install

```sh
agentstack add skill-kennethkhoocy-legal-scholarship-skills-download-gated-pdfs
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Download bot-gated PDFs via Wayback id_

## Problem
Many think-tank and publisher sites (taxpolicycenter.org, urban.org, SSRN delivery)
serve an HTML bot-challenge page instead of the PDF to non-browser clients. A browser
User-Agent header does not help. The downloaded "PDF" is actually HTML.

## Context / Trigger Conditions
- `curl -o file.pdf ` succeeds but the file starts with `id_/" -o out.pdf
   ```
   `` is any year likely to have a snapshot (e.g. publication year); Wayback
   redirects to the nearest capture. The `id_` suffix after the timestamp is what
   requests the untouched original.
2. Verify the download with pypdf — a bot page fails immediately:
   ```python
   from pypdf import PdfReader
   r = PdfReader("out.pdf"); print(len(r.pages), "pages")
   ```
3. If Wayback has no capture, fall back to: another mirror found via search
   (Exa/Firecrawl), or Firecrawl scrape for the parsed text when the binary is not
   strictly needed.

## Verification
`PdfReader` opens the file and reports a plausible page count; first-page text matches
the expected title.

## Example
Verified 2026-07-15: `taxpolicycenter.org/sites/default/files/publication/165884/ssrn-id4797771.pdf`
and `urban.org/sites/default/files/publication/80621/2000790-...pdf` both bot-gated to
direct curl (with UA), both downloaded intact via
`https://web.archive.org/web/2024id_/` and `.../web/2023id_/` (18 and 12 pages).

## Notes
- Government data hosts (e.g. `ticdata.treasury.gov`) are usually NOT gated — try direct
  curl first; Wayback is the fallback, not the default.
- Wayback captures can be stale for frequently-revised documents; check the snapshot date
  if currency matters.
- See also: the `pdf` skill (parsing/extraction after download) and
  `firecrawl:firecrawl-scrape` (parsed markdown when the binary is unnecessary).

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [kennethkhoocy](https://github.com/kennethkhoocy)
- **Source:** [kennethkhoocy/legal-scholarship-skills](https://github.com/kennethkhoocy/legal-scholarship-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-kennethkhoocy-legal-scholarship-skills-download-gated-pdfs
- Seller: https://agentstack.voostack.com/s/kennethkhoocy
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.