Zju Literature Downloader

ZJU Literature Downloader

This skill turns the verified workflow into a repeatable, legally scoped process for finding, downloading, and reading papers through the user's Zhejiang University Library / WebVPN access.

Boundaries

Use only the user's legitimate institutional access. Do not bypass paywalls, DRM, CAPTCHA, Cloudflare, publisher bot checks, or two-factor authentication. If a page asks for CAPTCHA, QR login, SMS/OTP, Cloudflare, publisher bot checks, or a security challenge, stop and ask the user to complete it in Chrome.

Avoid mass downloading. Work in small batches, preferably after the user confirms the paper list. Leave a clear audit trail of what was downloaded, from where, and whether supporting information was found.

Do not ask the user to paste institutional passwords, CAS credentials, OTP codes, recovery codes, or session tokens into chat or terminal. If the user offers a password, decline and use the handoff-login workflow instead.

Exception for ZJU CAS saved-login pages: if the user explicitly says that Chrome has already filled the ZJU CAS credentials and authorizes clicking the login/confirm button, the agent may click that button once on the ZJU CAS/WebVPN/institutional SSO page without reading, copying, or typing any credential. This exception does not apply to CAPTCHA, QR login, SMS/OTP, publisher bot checks, or any page outside the expected institutional login flow.

Do not inspect or export cookies, passwords, local storage, browser profiles, or session files. Use the browser's already-authenticated page context only.

Preconditions

Before attempting downloads, confirm these conditions:

1. Chrome is open on the user's machine.
2. The user has personally logged in to Zhejiang University Library / WebVPN in Chrome.

• Common pages: https://libweb.zju.edu.cn/, https://webvpn.zju.edu.cn/, https://zju.summon.serialssolutions.com/.

1. Chrome remote debugging is allowed for the current browser instance.

• Ask the user to open chrome://inspect/#remote-debugging.
• They must enable Allow remote debugging for this browser instance.

1. The environment can run Node.js 22+.

• Try node --version.
• If node is not on PATH in Codex Desktop, try %LOCALAPPDATA%\OpenAI\Codex\bin\node.exe.

1. The web-access CDP proxy is available or can be started.

• Typical Claude Code path: %USERPROFILE%\.claude\skills\web-access-main\scripts\check-deps.mjs.
• Typical shared agent path: %USERPROFILE%\.agents\skills\web-access-main\scripts\check-deps.mjs.
• In Codex-only setups also check %USERPROFILE%\.codex\skills\web-access-main\scripts\check-deps.mjs.

1. The user has approved the target output folder.

If Claude Code says this skill is not installed, install or copy it to:

$env:USERPROFILE\.claude\skills\zju-literature-downloader

Codex and other agent setups may instead use .codex\skills or .agents\skills; treat the three locations as install targets, not as different skill versions.

Batch Scope

Small batches are supported when the user provides a definite DOI/title/PMID list.

Recommended limits:

• normal batch: 5-10 papers
• upper practical batch: 15-20 papers, with pauses and a manifest
• stop immediately if publisher checks, CAPTCHA, WebVPN expiry, or unusual download prompts appear

Do not turn a broad keyword search into unlimited automatic downloading. Do not download whole journal issues, volumes, or large result sets.

Status Categories

Classify every paper into one of these statuses, and keep the status in the manifest:

downloaded
downloaded_with_si
cas_waiting_user
cas_resolved_retry_needed
publisher_verification_waiting_user
sciencedirect_robot_check
retry_after_user_verification
do_not_auto_retry
url_needs_repair
summon_no_link
publisher_blocked_waiting_user
no_authorized_pdf_found
failed_after_retry
ip_not_authorized
purchase_required

Use cas_waiting_user only when the browser is visibly at Zhejiang University CAS / unified identity authentication or an equivalent institutional SSO step. Do not treat this as a final failure.

Use publisher_verification_waiting_user or sciencedirect_robot_check when a publisher page shows "Are you a robot?", CAPTCHA, Cloudflare, bot verification, or another anti-automation challenge. Do not treat this as a final failure, but do not try to solve it automatically.

Start Chrome Control

Use the web-access CDP proxy when available.

On Windows PowerShell:

$node = "node"
if (-not (Get-Command node -ErrorAction SilentlyContinue)) {
  $node = "$env:LOCALAPPDATA\OpenAI\Codex\bin\node.exe"
}
$checkDepsCandidates = @(
  "$env:USERPROFILE\.claude\skills\web-access-main\scripts\check-deps.mjs",
  "$env:USERPROFILE\.agents\skills\web-access-main\scripts\check-deps.mjs",
  "$env:USERPROFILE\.codex\skills\web-access-main\scripts\check-deps.mjs"
)
$checkDeps = $checkDepsCandidates | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $checkDeps) { throw "web-access-main/scripts/check-deps.mjs not found" }
& $node $checkDeps

Then test:

Invoke-WebRequest -UseBasicParsing -Uri "http://127.0.0.1:3456/targets" -TimeoutSec 10

If this hangs or fails:

• Ask the user to confirm the remote debugging checkbox.
• Check %TEMP%\cdp-proxy.log.
• Do not attempt to read Chrome session files.

Core CDP Proxy API

The usual web-access proxy is http://127.0.0.1:3456.

| Endpoint | Method | Purpose | |----------|--------|---------| | /health | GET | Proxy health and Chrome connection status | | /targets | GET | List open Chrome tabs | | /new?url=... | GET | Create a new background tab | | /navigate?target=...&url=... | GET | Navigate an existing tab | | /close?target=... | GET | Close a tab | | /info?target=... | GET | Get page title, URL, and readyState | | /eval?target=... | POST body=JS | Execute JavaScript in a tab and return { value: ... } | | /clickAt?target=... | POST body=CSS selector | Click the center of a visible element |

Use /navigate rather than /new for Summon URLs containing #! fragments. If a #! fragment is stripped, Chrome may open about:blank or a wrong page. If curl is unavailable, use PowerShell Invoke-WebRequest for simple checks or Node.js fetch() for proxy calls.

Recommended Search Workflow

Prefer the library discovery route before direct publisher pages. It is more stable and less likely to trigger bot protection.

1. Search by DOI or exact title in ZJU Summon:

• https://zju.summon.serialssolutions.com/search?#!/search?pn=1&ho=t&include.ft.matches=f&l=en&q=

1. Open Summon URLs through scripts/cdp_open_url.mjs or otherwise fully URL-encode the nested URL when calling the CDP proxy. Summon uses #!; if the # fragment is stripped, Chrome may open about:blank or the wrong page.
2. Read the result page with /eval.
3. Extract links whose visible text or aria-label is:

• PDF
• 在线全文
• Full Text
• View PDF
• publisher-specific full-text entries.

1. Prefer the result's PDF link when present.
2. Open the PDF link in a new background tab through the CDP proxy.
3. If the publisher shows a security challenge, ask the user to complete it manually.
4. Once the PDF is visible in Chrome, use scripts/browser_pdf_downloader.mjs to save it from the authenticated browser context.

Summon Field Notes

When extracting PDF, online full text, or publisher links from Summon:

1. Skip empty href values. The first online-full-text match can be a facet toggle rather than a real link.
2. Treat Summon as a slow SPA: wait about 6 seconds after navigation, then retry link extraction up to 3 times with short pauses.
3. If Summon redirects to login.serialssolutions.com/sso/login and says the IP is outside the authorized range, record ip_not_authorized. In that case the user may need the ZJU RVPN/SSLVPN client, not only WebVPN in a browser page, because CDP-controlled Chrome tabs may still use the local network IP.
4. If Summon has no PDF but has an online-full-text link, follow that route before constructing a publisher URL manually.

Publisher-Specific Patterns

Use these as DOI-based repair hints only after confirming the paper identity. Do not claim access if the page or PDF is not actually verified.

ACS Publications

For DOI prefix 10.1021/...:

https://pubs.acs.org/doi/pdf/

ACS supporting information often follows:

https://pubs.acs.org/doi/suppl//suppl_file/_si_001.pdf

The verified example 10.1021/acs.biomac.4c00102 used bm4c00102_si_001.pdf.

Wiley

For DOI prefixes such as 10.1002/... or 10.1111/..., first navigate to the authenticated Wiley article page, then fetch from that page's own origin:

/doi/pdfdirect/?download=true

Do not hardcode onlinelibrary.wiley.com: Summon may authenticate through a subdomain such as advanced.onlinelibrary.wiley.com, and cross-origin fetches can fail. Also avoid navigating the tab directly to pdfdirect, because the browser may start a download and leave the page context unusable. Prefer page-context fetch().

Springer Nature

For DOI prefixes such as 10.1007/... and 10.1186/...:

https://link.springer.com/content/pdf/.pdf

This usually works after the institutional login is resolved. Use browser-context fetch if direct shell download returns 401, 403, or an HTML login page.

Nature Communications

For OA DOI patterns such as 10.1038/s41467-..., try:

https://www.nature.com/articles/.pdf

Extract `` from the DOI suffix and still verify the PDF header and text.

bioRxiv

For DOI prefix 10.1101/..., try:

https://www.biorxiv.org/content/v1.full.pdf

If version 1 returns 404, try the article page or a later version. Verify that the title matches.

Frontiers

For DOI prefix 10.3389/..., try:

https://www.frontiersin.org/articles//pdf

If it fails, open the article page first and use its visible PDF link.

RSC Publishing

For DOI prefix 10.1039/..., do not assume ZJU has authorized full-text access. If PDF links return 404 or purchase pages, inspect the article page and record no_authorized_pdf_found or purchase_required rather than retrying aggressively.

Publisher Verification and ScienceDirect

ScienceDirect and some publisher platforms may show "Are you a robot?", CAPTCHA, Cloudflare, bot verification, or similar checks after repeated direct DOI navigation or automated tab opening. These pages are security and anti-automation challenges, not ordinary login confirmations.

Reduce the chance of triggering them by using a conservative access pattern:

1. Prefer ZJU Summon / WebVPN / library 在线全文 links before direct doi.org -> publisher navigation.
2. Process ScienceDirect and other sensitive publishers one article at a time.
3. Keep a visible audit trail in the manifest; do not open many publisher tabs in parallel.
4. Wait for each page to settle before looking for Download PDF, View PDF, or PDF.
5. Reuse the same tab after the user completes a verification step instead of opening repeated new tabs.
6. Avoid retry loops. One failed automatic attempt is enough before handing the page to the user.

When a publisher verification page appears:

1. Stop automated actions on that tab.
2. Record the paper in publisher_verification.tsv or the main manifest with status publisher_verification_waiting_user; use sciencedirect_robot_check for ScienceDirect's "Are you a robot?" page.
3. Tell the user which paper and tab need manual attention.
4. Do not click CAPTCHA, Cloudflare, "Are you a robot?", bot-check, or similar challenge controls automatically.
5. After the user says the verification is complete, continue from the same tab and try the visible article/PDF route once.
6. If verification immediately reappears, mark do_not_auto_retry and move on.

ScienceDirect / Elsevier Field-Tested Workflow

ScienceDirect can show a "please wait" interstitial after CDP navigation. That page is not always a CAPTCHA. First wait briefly and inspect visible text: only treat it as sciencedirect_robot_check when it actually shows "Are you a robot?", CAPTCHA, Cloudflare, or another verification challenge.

For Elsevier papers, a practical workflow is:

1. Resolve institutional access once per browser session. From an article page, use "Access through Zhejiang University" or the Shibboleth handoff to ZJU CAS if needed.
2. After successful access, authenticated article pages often show Brought to you by: Zhejiang University Library.
3. For a modest manual-attention batch, open the remaining Elsevier DOI/article tabs, then notify the user that those tabs need manual clicks. The user handles interstitials, CAPTCHA, CAS, or "Access through Zhejiang University" buttons.
4. After the user confirms, scan tabs for article pages containing both Brought to you by and Zhejiang.
5. Find the main article View PDF link carefully. ScienceDirect pages may contain reference-section View PDF links far down the page. Normalize whitespace with a.textContent.replace(/\s/g, " ").trim() === "View PDF", then prefer the link nearest the top toolbar or matching the article PII.
6. When the user clicks View PDF, Chrome may open a new pdf.sciencedirectassets.com tab with a time-limited signed PDF URL. Download from that PDF tab with fetch(location.href, { credentials: "include" }) and close the PDF tab after success.
7. If the PDF tab returns Failed to fetch, the signed URL may have expired. Ask the user to re-click View PDF on the article tab.

This is still a human-in-the-loop workflow. Do not automate CAPTCHA, bot checks, or security prompts, and do not open very large ScienceDirect batches.

Create or update publisher_verification.tsv when publisher checks interrupt a batch. Use this header:

id	project	title	doi	year	venue	publisher	status	source_url	current_url	next_action	notes

Suggested next_action values:

user_complete_publisher_verification
retry_same_tab_after_user_confirms
try_summon_route
try_authorized_oa_route
mark_do_not_auto_retry

PowerShell example for opening a Summon URL safely:

$node = "$env:LOCALAPPDATA\OpenAI\Codex\bin\node.exe"
& $node "$env:USERPROFILE\.claude\skills\zju-literature-downloader\scripts\cdp_open_url.mjs" `
  --url "https://zju.summon.serialssolutions.com/search?#!/search?pn=1&ho=t&include.ft.matches=f&l=en&q=10.1021%2Facs.biomac.4c00102" `
  --wait

CAS SSO Handoff and Retry

Some publishers, especially Elsevier/ScienceDirect, Springer Nature, Nature Portfolio, Wiley, Taylor & Francis, Cell Press, and society platforms routed through Shibboleth/OpenAthens, may redirect to Zhejiang University CAS even when WebVPN is open. This is not a reason to ask for the user's password.

When a paper reaches a CAS or institutional SSO page:

1. Stop automated actions on that tab.
2. Record the paper in cas_retry.tsv with status cas_waiting_user.
3. Tell the user exactly which tab/page needs attention, for example: "This paper is at ZJU CAS. If Chrome has already filled the account and password, I can click the login/confirm button once with your authorization; otherwise please complete it in Chrome."
4. Do not read, store, or request the password, QR result, OTP, SMS code, CAPTCHA, cookie, or local/session storage.
5. If the user explicitly authorizes clicking because the CAS credentials are already filled in Chrome, click only the visible ZJU CAS/WebVPN/institutional SSO login/confirm button once. Do not type into fields or inspect hidden credential values.
6. If QR login, SMS/OTP, CAPTCHA, Cloudflare, or publisher bot verification appears, stop and let the user complete it manually.
7. After the login/confirm step completes, refresh or continue from the same tab.
8. Re-detect whether the page is now a publisher article page, a PDF viewer, or another institutional hand

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: baihe26
• Source: baihe26/zju-literature-downloader
• License: MIT

Install and usage instructions live in the source repository linked above.