Lineage Pii And Governance

Lineage, PII, And Governance

Overview

Governance is an engineering concern, not a cleanup exercise. This skill ensures that every meaningful data change accounts for ownership, lineage, access, and sensitive-data handling before release.

When to Use

• publishing a new table, model, or stream
• changing business-critical metrics
• handling personal, financial, health, or otherwise sensitive data
• modifying access controls or data-sharing patterns
• changing upstream or downstream lineage

Workflow

1. Identify ownership and consumers.

Every published dataset should have:

• an owner
• intended consumers
• known downstream dependencies

1. Classify the data.

Determine whether fields are:

• public
• internal
• confidential
• regulated or sensitive

1. Define required controls.

Controls may include:

• masking
• tokenization
• row-level restrictions
• column-level restrictions
• encryption requirements
• retention or deletion rules

1. Update lineage and documentation.

Record how the data flows from source to publish layer, including major transformations.

1. Verify policy enforcement in implementation.

Do not stop at documentation. Check that access and masking rules are actually reflected in code or platform configuration.

Common Rationalizations

| Rationalization | Reality | | --- | --- | | "It is only internal data." | Internal datasets still create exposure, misuse, and compliance risk. | | "We will document lineage later." | Lineage that is not updated during change work becomes stale immediately. | | "Security will handle masking downstream." | Sensitive data should be controlled as close to production as possible. |

Red Flags

• published data has no named owner
• sensitive fields are copied without classification
• lineage updates are missing for a shared metric
• access rules are assumed but not enforced

Verification

• [ ] Dataset ownership and consumers are identified
• [ ] Sensitive fields are classified
• [ ] Required controls are implemented or explicitly planned
• [ ] Lineage and documentation reflect the change
• [ ] Governance checks are based on real enforcement, not comments alone

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: vaquarkhan
• Source: vaquarkhan/data-engineering-agent-skills
• License: MIT
• Homepage: https://vaquarkhan.github.io/data-engineering-agent-skills/

Install and usage instructions live in the source repository linked above.