# Engineering Data Pipelines

> Data engineering knowledge reference covering Airflow, Dagster, Kafka Streams, Flink, dbt, and data quality patterns. Use when building data pipelines, ETL workflows, stream processing, or data quality checks.

- **Type:** Skill
- **Install:** `agentstack add skill-telagod-code-abyss-engineering-data-pipelines`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [telagod](https://agentstack.voostack.com/s/telagod)
- **Installs:** 0
- **Category:** [Cloud & Infrastructure](https://agentstack.voostack.com/c/cloud-infrastructure)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [telagod](https://github.com/telagod)
- **Source:** https://github.com/telagod/code-abyss/tree/main/skills/engineering-data-pipelines
- **Website:** https://www.npmjs.com/package/code-abyss

## Install

```sh
agentstack add skill-telagod-code-abyss-engineering-data-pipelines
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# 数据工程域 · Data Engineering

```
编排：Airflow(调度) | Dagster(资产) | Prefect(现代流)
流处理：Kafka Streams(嵌入式) | Flink(集群) | Spark Streaming
质量：Great Expectations | dbt tests | Soda Core
```

## 编排检查项

幂等(UPSERT/分区覆盖) | 增量(`WHERE updated_at > last_run`) | 事件驱动触发 | 跨 DAG 依赖 | 数据血缘(`ref()`/Asset deps)

## 流处理检查项

时间语义选择 | Watermark 乱序容忍 | 状态 TTL 防膨胀 | Checkpoint 间隔 | 端到端 Exactly-Once | 背压监控

## 质量检查项

分层验证(源→转换→目标) | 完整性+准确性+一致性 | 及时性阈值 | 加权评分 | 告警(Slack/PagerDuty)

工具对比、API 用法、质量维度详见 [references/details.md](references/details.md)

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [telagod](https://github.com/telagod)
- **Source:** [telagod/code-abyss](https://github.com/telagod/code-abyss)
- **License:** MIT
- **Homepage:** https://www.npmjs.com/package/code-abyss

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-telagod-code-abyss-engineering-data-pipelines
- Seller: https://agentstack.voostack.com/s/telagod
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.