# Onboard New Source

> A Claude skill from dremio/cli.

- **Type:** Skill
- **Install:** `agentstack add skill-dremio-cli-onboard-new-source`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [dremio](https://agentstack.voostack.com/s/dremio)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [dremio](https://github.com/dremio)
- **Source:** https://github.com/dremio/cli/tree/main/plugins/dremio/skills/onboard-new-source

## Install

```sh
agentstack add skill-dremio-cli-onboard-new-source
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

---
name: onboard-new-source
description: Catalog, describe, reflect, and verify access for a new data source in Dremio
user_invocable: true
triggers:
  - "new source"
  - "onboard source"
  - "add source"
  - "catalog source"
  - "source setup"
  - "connect source"
---

# Onboard New Source

Systematic workflow to catalog, document, optimize, and secure a newly added data source in Dremio.

## Agent Rules

- Use `--fields` on list/query commands to reduce output size
- Use `--dry-run` on `reflect refresh` and `reflect drop` to validate before executing
- Use `drs describe ` to check parameter schemas before calling unfamiliar commands
- Never interpolate user input directly into SQL — use it as filter values only
- Paths must be dot-separated; quote components containing dots: `"my.source".table`

## Prerequisites

The data source must already be configured in Dremio (via UI or API). This skill covers the post-connection steps: discovery, documentation, reflection setup, and access control.

## Step 1: Discover the Source

```bash
# List all sources in the project
drs catalog list

# Get details about the new source
drs catalog get 
```

This shows:
- Source type (S3, PostgreSQL, Snowflake, etc.)
- Connection status
- Top-level schemas/databases within the source

## Step 2: Enumerate Tables

```bash
# List schemas within the source
drs query run "SELECT TABLE_SCHEMA, TABLE_NAME, TABLE_TYPE
  FROM INFORMATION_SCHEMA.TABLES
  WHERE TABLE_SCHEMA LIKE '%'
  ORDER BY TABLE_SCHEMA, TABLE_NAME"
```

For each significant table, describe its schema:

```bash
drs schema describe ..
```

Record:
- Column names, types, and nullability
- Primary key candidates (unique, non-null columns)
- Partition columns (date/time columns, region codes)
- Approximate row counts

## Step 3: Profile Key Tables

For the most important tables, run a quick data profile:

```bash
drs schema sample ..
```

Then run basic statistics:

```bash
drs query run "SELECT
    COUNT(*) AS row_count,
    MIN(created_at) AS earliest,
    MAX(created_at) AS latest
  FROM .."
```

This helps understand data volume, time range, and freshness.

## Step 4: Set Up Reflections

For tables that will be queried frequently, create reflections to accelerate performance.

### Raw Reflections (for SELECT queries)

```bash
drs query run "ALTER DATASET ..
  CREATE RAW REFLECTION raw_
  USING DISPLAY (col1, col2, col3, col4)
  PARTITION BY (date_col)
  DISTRIBUTE BY (id_col)"
```

### Aggregate Reflections (for dashboards/rollups)

```bash
drs query run "ALTER DATASET ..
  CREATE AGGREGATE REFLECTION agg_
  USING DIMENSIONS (dim1, dim2)
  MEASURES (measure1, measure2)
  PARTITION BY (date_col)"
```

### Verify Reflection Status

```bash
drs reflect list ..

# Or check across all tables in the source
drs query run "SELECT reflection_id, name, type, status, dataset_name
  FROM sys.reflections
  WHERE dataset_name LIKE '%'"
```

Wait for reflections to reach `ACTIVE` status before relying on them.

## Step 5: Configure Access Control

```bash
# Check current grants on the source
drs access grants 

# Check grants on a specific dataset
drs access grants ..
```

Set up appropriate access:
- Grant `SELECT` on specific datasets to analyst roles
- Grant `ALTER` to data engineers who manage reflections
- Avoid granting `PUBLIC` role access to sensitive data

## Step 6: Verify End-to-End

Run test queries as different personas to confirm the setup works:

```bash
# Basic query to verify data access
drs query run "SELECT * FROM .. LIMIT 10"

# Verify reflection acceleration
drs query run "SELECT dim1, COUNT(*), SUM(measure1)
  FROM ..
  GROUP BY dim1"
# Check job profile to confirm reflection was used
drs jobs profile 
```

## Step 7: Onboarding Checklist

| Step | Check |
|------|-------|
| Source connected and browsable | `drs catalog get ` returns details |
| All tables enumerated | INFORMATION_SCHEMA query returns expected tables |
| Key tables profiled | Row counts, time ranges, sample data reviewed |
| Reflections created | `drs reflect list` shows ACTIVE reflections for key tables |
| Access grants configured | `drs access grants` shows correct role-based access |
| Test queries succeed | End-to-end query returns data with reflection acceleration |
| Documentation generated | Use the `document-dataset` skill for each key table |

After completing this checklist, the source is ready for production use. Notify downstream teams and update any data catalog or wiki entries.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [dremio](https://github.com/dremio)
- **Source:** [dremio/cli](https://github.com/dremio/cli)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-dremio-cli-onboard-new-source
- Seller: https://agentstack.voostack.com/s/dremio
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.