Api And Saas Ingestion Patterns
Guides agents through API and SaaS ingestion workflows. Use when extracting data from REST, GraphQL, or SaaS platforms with pagination, rate limits, auth rotation, backfills, or unstable source contracts.
Install
$ agentstack add skill-vaquarkhan-data-engineering-agent-skills-api-and-saas-ingestion-patterns ✓ scanned · ✓ verified, works with Claude Code, Cursor, and more.
Security review
✓ PassedNo issues found. Passed automated security review. · v0.1.0 How review works →
- ✓ Prompt-injection patterns
- ✓ Secret / credential exfiltration
- ✓ Dangerous shell & filesystem operations
- ✓ Untrusted network calls
- ✓ Known-malicious package signatures
What it can access
- ✓ Network access No
- ✓ Filesystem access No
- ✓ Shell / process execution No
- ✓ Environment & secrets No
- ✓ Dynamic code execution No
From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.
Verified badge
Passed review? Show it. Paste this badge into your README, it links to the public security report.
Links to your public security report.[](https://agentstack.voostack.com/security/report/skill-vaquarkhan-data-engineering-agent-skills-api-and-saas-ingestion-patterns) Reliability & compatibility
Declared compatibility
Compatibility is declared by the source manifest. End-to-end runtime verification is coming, see below.
We're building live execution health for every listing: tool-call success rate, median latency, uptime, and last-checked timestamps, measured, not self-reported. It isn't live yet, so we don't show numbers we can't stand behind.
How agent discovery & health will work →About
API And SaaS Ingestion Patterns
Overview
Use this skill when the source system is an external API or SaaS platform rather than a database or file drop. It helps agents design reliable extraction, pagination, throttling, auth handling, and backfill-safe ingestion contracts.
When to Use
- extracting from REST or GraphQL APIs
- ingesting SaaS platform data
- handling pagination, cursors, or sync tokens
- dealing with rate limits and source-side throttling
- backfilling historical API data safely
Do not treat APIs like static tables. They change behavior, availability, and limits over time.
Workflow
- Define the source contract.
Include:
- endpoint or object name
- auth method
- extraction window
- pagination style
- rate limits
- data freshness expectations
- Design for extraction resilience.
Handle:
- retries
- backoff
- token refresh
- idempotent re-fetch behavior
- partial page failure
- Make incremental behavior explicit.
Decide whether the source supports:
- updated timestamps
- cursors
- change tokens
- full snapshots only
- Record raw source evidence where useful.
API sources often need raw response retention for incident diagnosis.
- Validate contracts against source drift.
Common Rationalizations
| Rationalization | Reality | | --- | --- | | "We can just loop through pages." | Pagination bugs and retry gaps often cause silent data loss. | | "The vendor API is stable enough." | SaaS APIs change rate limits, fields, and semantics more often than teams expect. | | "If a request fails, we can rerun later." | Without windowing and idempotency rules, reruns can miss or duplicate data. |
Red Flags
- no rate-limit strategy exists
- extraction windows depend on undocumented source behavior
- retries ignore duplicate or partial-page risks
- auth rotation and token expiry are not considered
Verification
- [ ] The source contract covers pagination, limits, auth, and cadence
- [ ] Extraction retries and failures are operationally safe
- [ ] Incremental or snapshot behavior is explicit
- [ ] Source drift and raw evidence handling are considered
Source & license
This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.
- Author: vaquarkhan
- Source: vaquarkhan/data-engineering-agent-skills
- License: MIT
- Homepage: https://vaquarkhan.github.io/data-engineering-agent-skills/
Install and usage instructions live in the source repository linked above.
Reviews
No reviews yet, be the first.
Write a review
Versions
- v0.1.0 Imported from the upstream source.