# Spark Lake Catalog

> Spark-specific patterns for reading and writing oleander lake catalog tables, including append vs overwrite, avoiding driver writes, and reusable table naming.

- **Type:** Skill
- **Install:** `agentstack add skill-oleanderhq-claude-plugin-spark-lake-catalog`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [OleanderHQ](https://agentstack.voostack.com/s/oleanderhq)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [OleanderHQ](https://github.com/OleanderHQ)
- **Source:** https://github.com/OleanderHQ/claude-plugin/tree/main/skills/spark-lake-catalog

## Install

```sh
agentstack add skill-oleanderhq-claude-plugin-spark-lake-catalog
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# oleander Spark Lake Catalog

Use this skill when reading from or writing to the oleander lake catalog in a Spark job.

For shared catalog conventions such as table naming, namespaces, and avoiding raw storage paths, also use `lake-catalog`.

## Reading tables

Use `spark.table()` with the fully qualified table name:

```python
df = spark.table("oleander.default.sf_311")
```

Do not construct raw storage paths for Iceberg tables. Use catalog-qualified names so Spark reads the table through the Iceberg catalog.

## Writing tables

**Append** (add rows to an existing or new table):

```python
df.writeTo("oleander.my_namespace.my_table").append()
```

**Overwrite** (replace table contents):

```python
df.write.mode("overwrite").saveAsTable("oleander.my_namespace.my_table")
```

Use `writeTo(...).append()` for incremental pipelines. Use `write.mode("overwrite").saveAsTable(...)` when replacing the full result set each run.

## Prefer Spark writes over driver writes

Avoid collecting data to the driver and then writing from Python memory. Keep writes as Spark DataFrame operations so Iceberg handles the transaction, partitioning, and metadata correctly.

Bad:

```python
rows = df.collect()
# write rows from Python memory
```

Good:

```python
df.write.mode("overwrite").saveAsTable("oleander.my_namespace.my_table")
```

## Parameterize table names

Accept table names as arguments or environment variables so scripts are reusable:

```python
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("--input-table", default="oleander.default.sf_311")
parser.add_argument("--output-catalog", default="oleander.my_namespace")
args = parser.parse_args()

df = spark.table(args.input_table)
df.write.mode("overwrite").saveAsTable(f"{args.output_catalog}.results")
```

## Cache reused tables, then unpersist

If a table is read and used in multiple downstream transforms, cache it once and unpersist when done:

```python
df = spark.table("oleander.default.sf_311")
df.cache()
# ... multiple transforms ...
df.unpersist()
```

Do not cache tables that are only used once.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [OleanderHQ](https://github.com/OleanderHQ)
- **Source:** [OleanderHQ/claude-plugin](https://github.com/OleanderHQ/claude-plugin)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-oleanderhq-claude-plugin-spark-lake-catalog
- Seller: https://agentstack.voostack.com/s/oleanderhq
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.