# Review And Promote Traces

> Judge flagged trace outputs and promote judged items back to datasets. Use when an evaluation run requires human judgment or when review queue items need to be judged for promotion into the dataset.

- **Type:** Skill
- **Install:** `agentstack add skill-goodeye-labs-truesight-mcp-skills-review-and-promote-traces`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [Goodeye-Labs](https://agentstack.voostack.com/s/goodeye-labs)
- **Installs:** 0
- **Category:** [Developer Tools](https://agentstack.voostack.com/c/developer-tools)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [Goodeye-Labs](https://github.com/Goodeye-Labs)
- **Source:** https://github.com/Goodeye-Labs/truesight-mcp-skills/tree/main/skills/review-and-promote-traces
- **Website:** https://truesight.goodeyelabs.com/docs/mcp-integration

## Install

```sh
agentstack add skill-goodeye-labs-truesight-mcp-skills-review-and-promote-traces
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Review and Promote Traces

Use this skill for the judgment and promotion loop after trace evaluation.

## Interactive Q&A protocol (mandatory)

BEFORE the first scoping question, search for a structured question tool (e.g., `AskUserQuestion` or similar interactive widget) and load it. Use that tool for EVERY scoping question. Fall back to plain-text lettered options ONLY if no such tool exists in the environment.

If context is unclear, ask one question at a time using the structured question tool (loaded per the HARD-GATE above).

Example question structure:

```
What do you want to review first?
A) One specific run
B) Pending queue triage across runs
```

Rules:
- Ask one question per message.
- Use the structured question tool for every question. Structure each with a short header, 2-4 options with labels and descriptions, and place the recommended option first. Do not add "(Recommended)" or similar annotations to option labels.
- Ask one follow-up when needed, then continue.

## Workflow

1. If needed, flag evaluated run:
   - Use `flag_review_item` with `run_id`.
2. Retrieve review items:
   - Use `list_review_items` with pagination.
3. Collect human judgments:
   - Present review items to the user and ask for judgment one item at a time when needed.
   - Capture `judgment_value` and optional `notes` for each item.
   - Use the structured question tool (loaded per the HARD-GATE above) to present options that match the live evaluation setup:
     - Binary example:
       - A) Pass
       - B) Fail
     - Categorical example:
       - A) 
       - B) 
       - C) 
     - Continuous example:
       - A) Enter numeric value (within configured range)
       - B) Skip this item for now
   - If valid options are unclear, look them up before asking:
     1) Inspect `list_review_items` payload for item result details and expected value hints.
     2) Use `get_result(run_id)` for run-level context and chain outputs.
     3) If evaluation id is available in run metadata, call `get_evaluation(evaluation_id)` and read config/judgment criteria to derive valid judgment options.
   - When options remain ambiguous after lookup, ask one clarification question using the structured question tool before proceeding.
4. Submit judgments:
   - For each target item, call `judge_review_item`.
   - Pass the user-provided `judgment_value` and optional `notes` into `judge_review_item`.
   - Include notes when judgment context matters.
5. Promote judged outputs:
   - Use `add_reviewed_items_to_dataset` with `run_id`.
6. Report result:
   - number of items judged
   - promotion status and row counts
   - any skipped or blocked items

## Queue-wide triage guidance

- Group by run and status first.
- Prioritize high-impact runs or oldest pending runs.
- Keep an audit trail of judgment rationale in notes.

## Scopes reference

- `list_review_items` requires `review:read`
- `flag_review_item`, `judge_review_item`, and `add_reviewed_items_to_dataset` require `review:write`

If a scope error occurs, ask the user to create a key with the missing scope in Truesight Settings.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [Goodeye-Labs](https://github.com/Goodeye-Labs)
- **Source:** [Goodeye-Labs/truesight-mcp-skills](https://github.com/Goodeye-Labs/truesight-mcp-skills)
- **License:** MIT
- **Homepage:** https://truesight.goodeyelabs.com/docs/mcp-integration

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-goodeye-labs-truesight-mcp-skills-review-and-promote-traces
- Seller: https://agentstack.voostack.com/s/goodeye-labs
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.