# Shadow Frog Dream

> >-

- **Type:** Skill
- **Install:** `agentstack add skill-microsoft-shadowfrog-shadow-frog-dream`
- **Verified:** Pending review
- **Seller:** [microsoft](https://agentstack.voostack.com/s/microsoft)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [microsoft](https://github.com/microsoft)
- **Source:** https://github.com/microsoft/ShadowFrog/tree/main/skills/shadow-frog-dream

## Install

```sh
agentstack add skill-microsoft-shadowfrog-shadow-frog-dream
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# ShadowFrog Dream

Autonomous experimentation while the user is away. Every task is an
**experiment** — implement real code in a worktree, run it, persist as a
**named git branch** pushed to the fork. Dream's unique value is
implementation experience that **compounds across sessions**.

## Critical Invariants

These rules are stated ONCE here and enforced by helper scripts. Violating
any of them is a completion criteria failure.

### Prerequisite: `.shadow/` must be git-tracked

Dream moves `.shadow/` content **through git** — artifacts are committed onto
the dream branch, pushed to the remote, then read back by the reconciler via
`git show origin/ .shadow/...`. If `.shadow/` is gitignored (the
"local only" option in `shadow-frog-init`), `git add -A` silently skips those
files, nothing reaches the remote, and the reconciler finds no manifest —
**every discovery is lost without warning**. `dream-setup.sh` runs
`git check-ignore .shadow` up front and refuses to start if it's ignored.
Use `shadow-frog-update` instead for local-only shadows.

### Path Isolation

```
WORKTREE_BASE = /tmp/shadowfrog-dreams//
WORKTREE_DIR  = $WORKTREE_BASE/dream-
```

- Worktrees are ALWAYS in `/tmp/shadowfrog-dreams//`, NEVER in
  the project directory. This prevents conflicts between parallel agents
  and keeps the main repo clean.
- `DREAM_NS` (namespace) isolates branches per task/instance. Resolved
  from: `DREAM_NAMESPACE` env → `TASK_INFO.json` → `.env` → repo basename.
- Override only with `DREAM_WORKTREE_BASE` env var if `/tmp` is too small.
- `dream-setup.sh` computes and enforces all paths. Use it.

### Branch Naming

```
BRANCH_NAME = dream//
DREAM_ID    = YYYYMMDD-HHMMSSZ-
```

### Artifact Format

```
.shadow/_dreams//report.md
.shadow/_dreams//manifest.json
.shadow/_dreams//patch.diff
```

NEVER flat files (`_dreams/.md`). Flat files break the pipeline.

### RUN_PREFIX

All python/pytest commands MUST use the `RUN_PREFIX` resolved during
preflight. When `RUN_PREFIX="uv run"`, use `$RUN_PREFIX python3 ...`.
Bare `python3` or `pytest` without prefix is a violation when non-empty.

### Reconciliation is Mandatory

Every dream branch must reconcile to main before the session ends. The
most common failure mode is agents pushing dream branches but never
reconciling — losing all discoveries.

**Two modes:**
- **Parallel mode (default):** Launch a batch of 3-4 sub-agents → wait
  for all to push → run `dream-reconcile.py "$REPO_ROOT"` ONCE at the end
  of the batch. The reconciler auto-discovers every un-reconciled dream
  branch in the namespace — you do NOT pass branch names. One call merges
  every pushed branch.
- **Sequential mode (fallback when sub-agents unavailable):**
  Complete dream → push → reconcile → verify → next dream. Adds ~30s
  per dream but guarantees zero data loss if the session crashes
  mid-batch.

Never queue multiple un-reconciled batches; reconcile at the end of
each batch or each individual dream.

### Script Failure Recovery

All helper scripts (`dream-setup.sh`, `dream-reconcile.py`,
`dream-validate.py`) are **self-documenting**. If a script fails or is
unavailable: **read the script source**, understand what it does, and adapt
its logic manually for your situation. Never skip steps just because a
script errored — the steps still need to happen.

```
Fork repo (user/target-repo)
  main --- .shadow/ (accumulated ALL discoveries)
  |
  +-- dream//  (cycle 1, agent A, from main)
  +-- dream//  (cycle 1, agent B, from main)
  +-- dream//  (cycle 2, from dream//, compounding)
```

- Branches are live — `git checkout dream/` runs the code
- Shadow follows lineage — ancestor chain, not sibling branches
- Main is the accumulator — reconciliation merges ALL discoveries

### Prerequisites

1. Forked repo cloned locally with pushable remote
2. ShadowFrog skills installed (`install.sh --project /path/to/fork`)
3. `.shadow/` initialized (`/shadow-frog-init`)

## Helper Scripts

This skill bundles 6 helper scripts. Find them in the skill directory:

```bash
SKILL_DIR=""
for DIR in .github/skills/shadow-frog-dream .claude/skills/shadow-frog-dream; do
    [ -d "$DIR" ] && SKILL_DIR="$DIR" && break
done
```

| Script | Purpose | When to use |
|--------|---------|-------------|
| `dream-setup.sh` | Creates worktree + branch with namespace isolation | **Phase 3** — start of every experiment |
| `dream-validate.py` | Validates artifacts before push (hard gate) | **Phase 5** — before `git push` |
| `dream-reconcile.py` | Merges dream branches into main's `.shadow/` | **Phase 6** — after all experiments done |
| `dream-coverage.py` | Computes exploration coverage map | **Phase 2** — task planning for diversity |
| `dream-cleanup.sh` | Safely removes ONE dream worktree (with safety gate) | **After push** — replaces the old inline cleanup snippet |
| `dream-gc.sh` | Sweeps orphan dream worktrees from `$DREAM_WORKTREE_BASE` | **Auto** — triggered by `dream-setup.sh` (per-namespace throttle, default 1× / hour) in orphan-only mode; also `--task-complete --namespace "$DREAM_NS" --min-age-min 0` for end-of-session sweep of registered-but-stale dirs |

**Usage patterns:**

```bash
# Setup: creates worktree, prints export vars.
# IMPORTANT: capture the output FIRST, then eval it. Writing
# `eval "$(dream-setup.sh ...)" || exit 1` does NOT catch failures: if the
# command substitution exits non-zero and prints nothing, `eval ""` still
# succeeds (exit 0) and the agent silently proceeds with empty env vars.
# Assigning to a variable makes `|| exit 1` fire on the script's real exit code.
SETUP_OUT="$("$SKILL_DIR/dream-setup.sh" --slug t01-my-experiment)" || exit 1
eval "$SETUP_OUT"
# → exports (keep in sync with dream-setup.sh emit_export block):
#   REPO_ROOT, DEFAULT_BRANCH, DREAM_NS, DREAM_ID, BRANCH_NAME, PARENT_BRANCH,
#   WORKTREE_DIR, WORKTREE_BASE, BASE_COMMIT, RUN_PREFIX, SLUG

# Validate: hard gate before push
python3 "$SKILL_DIR/dream-validate.py" "$DREAM_ID" "$WORKTREE_DIR"

# Reconcile: merge all dream branches into main
python3 "$SKILL_DIR/dream-reconcile.py" "$REPO_ROOT"
# After `git push` succeeds, optionally clean up reconciled branches.
# Cleanup REFUSES to run if `.shadow/` has uncommitted changes, or unless
# HEAD is already on origin/ — so the canonical flow is:
#   reconcile → git add .shadow/ && git commit && git push → re-run --cleanup-branches
python3 "$SKILL_DIR/dream-reconcile.py" "$REPO_ROOT" --cleanup-branches

# Coverage: show which files still need exploration
python3 "$SKILL_DIR/dream-coverage.py" "$REPO_ROOT"

# All scripts support --help.
```

If a script is not found or fails, read its source — they are
self-documenting. Adapt the steps manually if needed (see each phase for
inline fallback instructions).

## Phase 1: Preflight and Assess

### Preflight Validation

```bash
REPO_ROOT=$(git rev-parse --show-toplevel)
cd "$REPO_ROOT"

# 0. Auto-detect DREAM_NAMESPACE
if [ -z "${DREAM_NAMESPACE:-}" ]; then
    if [ -f TASK_INFO.json ]; then
        DREAM_NAMESPACE=$(python3 -c "import json; print(json.load(open('TASK_INFO.json')).get('dream_namespace',''))" 2>/dev/null)
        export DREAM_NAMESPACE
    elif [ -f .env ]; then
        DREAM_NAMESPACE=$(grep '^DREAM_NAMESPACE=' .env | head -1 | cut -d'=' -f2-)
        export DREAM_NAMESPACE
    fi
fi
[ -n "${DREAM_NAMESPACE:-}" ] && echo "Dream namespace: $DREAM_NAMESPACE"

# 1. Detect default branch
DEFAULT_BRANCH=$(git symbolic-ref refs/remotes/origin/HEAD 2>/dev/null | sed 's|refs/remotes/origin/||')
if [ -z "$DEFAULT_BRANCH" ]; then
    if git show-ref --verify refs/remotes/origin/main >/dev/null 2>&1; then
        DEFAULT_BRANCH="main"
    elif git show-ref --verify refs/remotes/origin/master >/dev/null 2>&1; then
        DEFAULT_BRANCH="master"
    else
        echo "ERROR: Cannot detect default branch. Fix: git remote set-head origin "
    fi
fi
echo "Default branch: $DEFAULT_BRANCH"

# 2-5. Validate environment
echo "Current branch: $(git branch --show-current)"
echo "Remote: $(git remote get-url origin)"
git ls-remote origin HEAD >/dev/null 2>&1 || echo "ERROR: Cannot reach remote."
[ -d .shadow ] || echo "ERROR: .shadow/ not found. Run /shadow-frog-init first."
mkdir -p .shadow/_dreams
git diff --quiet && git diff --cached --quiet || echo "ERROR: Uncommitted changes."

# 6. Fetch all remote branches (ONE fetch for all agents)
git fetch origin --prune

# 7. List dream branches (namespace-filtered)
DREAM_NS="${DREAM_NAMESPACE:-}"
BRANCH_PATTERN="${DREAM_NS:+origin/dream/${DREAM_NS}/}"
BRANCH_PATTERN="${BRANCH_PATTERN:-origin/dream/}"
echo "Available dream branches:"
git branch -r | grep "$BRANCH_PATTERN" | sed 's|origin/||' || echo "  (none)"

# 8. Detect RUN_PREFIX from lockfiles
if [ -f uv.lock ]; then
    uv sync --all-groups 2>&1 | tail -3
    RUN_PREFIX="uv run"
elif [ -f package-lock.json ]; then
    npm install --quiet 2>&1 | tail -3
    RUN_PREFIX="npx"
elif [ -f yarn.lock ]; then
    yarn install --silent 2>&1 | tail -3
    RUN_PREFIX="npx"
else
    RUN_PREFIX=""
fi
echo "RUN_PREFIX='$RUN_PREFIX'"

# 9. List compoundable experiments
echo ""
echo "=== COMPOUNDABLE EXPERIMENTS ==="
if [ -f .shadow/_dreams/_index.md ]; then
    awk -F'|' 'NR>2 && /useful/ {
        gsub(/ /,"",$2); gsub(/ /,"",$4); gsub(/ /,"",$6);
        gsub(/^ +| +$/,"",$5);
        if ($2 != "" && $6 != "") print $6 " | " $3 " | " $5
    }' .shadow/_dreams/_index.md
    COMPOUNDABLE=$(awk -F'|' 'NR>2 && /useful/ {gsub(/ /,"",$2); if ($2 != "") c++} END {print c+0}' .shadow/_dreams/_index.md)
    echo "Total compoundable: $COMPOUNDABLE"
else
    echo "(none — first dream session)"
fi
```

**If any check prints ERROR, STOP.** Do not use `exit 1` — check output
and stop at the agent level.

(`RUN_PREFIX` MUST be threaded into every subagent prompt — see Critical
Invariants above. Bare `python3`/`pytest` without prefix = completion
criteria violation.)

### Snapshot Branch State

After the single `git fetch`, capture dream branches and pass to all
sub-agents — they do NOT fetch independently.

```bash
DREAM_NS="${DREAM_NAMESPACE:-}"
BRANCH_FILTER="${DREAM_NS:+origin/dream/${DREAM_NS}/}"
BRANCH_FILTER="${BRANCH_FILTER:-origin/dream/}"
git branch -r --format='%(refname:short) %(objectname:short)' \
  | grep -F "$BRANCH_FILTER" \
  | sed 's|origin/||' > .shadow/_dreams/.branch-map.txt
cat .shadow/_dreams/.branch-map.txt

# Initialize session tracking (orchestrator-only; agents do NOT write here)
: > .shadow/_dreams/.session-branches.txt
```

### Assess Codebase

Read `_meta/state.json`, `_index.md`, existing discoveries, and **past
dream reports** in `_dreams/`.

### Build Exploration Coverage Map

File-level coverage breadth is the strongest predictor of dream success
(r²=0.63 vs bugs found), NOT dream count (r²=0.04).

```bash
# Find and run the coverage script
COVERAGE_SCRIPT=""
for DIR in .github/skills/shadow-frog-dream .claude/skills/shadow-frog-dream; do
    [ -f "$DIR/dream-coverage.py" ] && COVERAGE_SCRIPT="$DIR/dream-coverage.py" && break
done
[ -n "$COVERAGE_SCRIPT" ] && python3 "$COVERAGE_SCRIPT" "$REPO_ROOT" || echo "WARNING: dream-coverage.py not found"
```

**Coverage definition:** A file is "covered" only when its shadow has
≥1 behavioral discovery (line starting with `- `). Placeholder-only = NOT covered.

**Scoped exploration (`--scope`)** — pass `--scope ` (repeatable)
to restrict the coverage map to a specific subtree. Use this when the
broader repo is well-explored but a particular area (e.g., a known
frontier of bugs, a newly-added module, a subsystem the user just
flagged) deserves a focused dream session. All counts (totals, %,
saturated, fan-in, per-dir) are computed over the scoped subset only.

```bash
# Scope to one subtree
python3 "$COVERAGE_SCRIPT" "$REPO_ROOT" --scope src/auth/

# Scope to multiple subtrees in one pass
python3 "$COVERAGE_SCRIPT" "$REPO_ROOT" --scope src/auth/ --scope src/db/
```

When using `--scope`, the per-category task quotas (Phase 2) still apply
but are interpreted against the scoped subset. Don't use scoped
exploration as the default — pick it only when there's a concrete reason
to concentrate effort. Unscoped diversity remains the strongest
predictor of useful discoveries.

### Review Past Dreams (Required)

When compoundable experiments exist (preflight step 9):
- Read each report: `cat .shadow/_dreams//report.md`
- Choose which to continue (extending, fixing, integrating)
- Note `dead_end` experiments to avoid repeating
- Trace lineage via the `parent` column in `_dreams/_index.md`

**Compounding quality gate** — before choosing to compound from a parent:

1. Read the parent's `report.md` AND `manifest.json`
2. Verify the parent has a non-empty `patch.diff` (prose-only parents
   are low-value — prefer parents with working code)
3. Identify at least one specific file or function you plan to modify/extend
4. Check the parent's area isn't saturated (8+ discoveries) — if it is,
   start fresh from main unless you have a concrete new angle
5. Log your compounding intent: "I will extend parent's retry logic in
   `src/http.py` to handle connection timeouts" — vague "continue
   exploring" is NOT compounding

**First dream session:** if preflight step 9 shows `(none)`, all tasks
branch from main.

## Phase 2: Plan

Generate a concrete plan. **Target 12 tasks (2 per category).** On small
codebases (/ (compounds prior)
       Target: src/parsers/csv.py (extending parent's failing tests)
       Why: Parent found 2 crashes, need to verify fixes
  ...
```

### Diversity Rules

Prevent fixation (exploring the same files while leaving most untouched):

1. **Max 2 tasks per source file** (unless prior dream left concrete follow-up)
2. **≥30% of tasks on uncovered files** (from coverage map)
3. **≥2 tasks on "deep" files** (utilities, internals, converters)
4. **Vary directories** — no 3+ consecutive tasks in same dir

**Self-check before finalizing:** unique target files ≥ 60% of task count,
uncovered file tasks ≥ 30%, no file in > 2 tasks. Swap if failing.

**Escape hatches** (document justification): prior dream's failing test,
concrete untested hypothesis, file is 500+ lines with unexplored sections,
codebase has /)" || exit 1
eval "$SETUP_OUT"
```

Capture into `SETUP_OUT` first, then `eval` it — see Helper Scripts §
usage patterns (above) for why bare `eval "$(…)" || exit 1` silently
swallows the script's exit code.

This exports (keep in sync with `dream-setup.sh`): `REPO_ROOT`,
`DEFAULT_BRANCH`, `DREAM_NS`, `DREAM_ID`, `BRANCH_NAME`, `PARENT_BRANCH`,
`WORKTREE_DIR`, `WORKTREE_BASE`, `BASE_COMMIT`, `RUN_PREFIX`, `SLUG`.

**If `dream-setup.sh` fails or is not found:** Apply the Script Failure
Recovery rule (read the script source, adapt its logic). Common causes:
missing git remote, branch already exists, `/tmp` permissions.

**Note:** Shell variables don't persist across tool calls. Either run
multi-step setup in a single shell, or re-derive values. From inside a
worktree, get main repo with:
`git -C "$(git rev-parse --git-common-dir)/.." rev-parse --show-toplevel`

**If worktree creation fails:** mark task `blocked`, replace with another.

### What Meaningful Compounding Looks Like

Compounding means **actively engaging with the parent's code**, not just
sitting on its branch. Valid compounding approaches:
- **Extend**: import or call the parent's modules and build on them
- **Modify**: edit the parent's code to fix limitations noted in its report
- **Refactor**: restructure the parent's implementation for better design
- **Integrate**: wire the parent's standalone module into the real codebase
- **Test deeper**: add edge-case tests for the parent's implementation

Don't assume the parent dream's code is complete or frozen — iterative
improvement is the whole point. If you can't find anything meaningful to
build on, start fresh from main instead.

Compounding that only adds a new standalone module beside the parent's
code (with no imports, edits, or integration) is NOT compounding — it's
a fresh experim

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [microsoft](https://github.com/microsoft)
- **Source:** [microsoft/ShadowFrog](https://github.com/microsoft/ShadowFrog)
- **License:** MIT

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** yes
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: flagged — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-microsoft-shadowfrog-shadow-frog-dream
- Seller: https://agentstack.voostack.com/s/microsoft
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.