# Monocle Test Gen

> Generate a pytest test file that uses the Monocle Test Tools fluent API (`monocle_trace_asserter`) to drive an agent under instrumentation and assert against the emitted traces. USE FOR: any request phrased as 'write/create/generate a Monocle test', 'add a fluent test for <agent>', 'test that <agent> calls <tool> with <input>', 'validate that <agent> produces <output>', or any description that na…

- **Type:** Skill
- **Install:** `agentstack add skill-monocle2ai-skills-monocle-test-gen`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [monocle2ai](https://agentstack.voostack.com/s/monocle2ai)
- **Installs:** 0
- **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [monocle2ai](https://github.com/monocle2ai)
- **Source:** https://github.com/monocle2ai/skills/tree/main/skills/monocle-test-gen

## Install

```sh
agentstack add skill-monocle2ai-skills-monocle-test-gen
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Generate a Monocle fluent test

You are translating a natural-language description into a runnable pytest file that uses `monocle_trace_asserter` (the fluent API from `monocle_test_tools`) to (1) run an agent under Monocle instrumentation and (2) assert against the traces it emits.

## Workflow

0. **Load the test plan.** Before anything else, look for a `{project}_test_plan.md` file in the working directory — where `{project}` is the project/agent name (e.g. `lg_travel_agent_test_plan.md`), falling back to a plain `test_plan.md` if no project-prefixed file exists. Use Bash `ls`/Glob to find it (e.g. `ls *_test_plan.md test_plan.md 2>/dev/null`). If found, READ it and treat it as the primary source for the three things below — the agent topology, exact agent/tool names, inputs, and validation scenarios are usually already enumerated there (e.g. in scenario tables grouped by ID). The user's request then selects WHICH scenario(s) from the plan to generate as tests. If no test plan file exists, fall back to extracting everything from the user's description.

1. **Extract three things from the test plan (preferred) or the user's description.** If any is missing or ambiguous, ASK before generating code — guessing here produces tests that compile but assert the wrong thing.
   - **Agent under test** — the Python import path of the agent/crew/workflow object, AND which framework it belongs to (maps to an `agent_type` string — see table below).
   - **Input** — the prompt/message/dict passed to the agent. Preserve it verbatim from the test plan / description.
   - **What to validate** — one or more of: a tool was called (with optional input/output substring), an agent was invoked, the final output contains/equals a string, token/duration limit, an Okahu/BERTScore eval result.

2. **Pick the framework runner string** from the agent's framework:

   | `agent_type` string | Framework |
   |---|---|
   | `google_adk` | Google ADK |
   | `langgraph` | LangGraph |
   | `crewai` | CrewAI |
   | `llamaindex` | LlamaIndex |
   | `strands` | Strands Agents |
   | `openai` | OpenAI Agents |
   | `msagent` | Microsoft Agent Framework / Semantic Kernel |

3. **Decide sync vs async.** Default to `async` + `run_agent_async` + `@pytest.mark.asyncio` unless the user explicitly says the agent is synchronous. Almost every framework runner in this repo is async.

4. **Locate the file.** Place new tests under `test_tools/tests/integration/` with filename `test__fluent.py`. If a file matching the framework + agent already exists there, ADD a new `async def test_...` function to it rather than creating a duplicate file. Use Bash `ls` / Grep to check first.

5. **Generate the test** following the template below. Then run it (or at least `pytest --collect-only` the file) to confirm it parses, unless the user said not to.

## Test template

```python
import pytest
from monocle_test_tools import TraceAssertion
from  import 

@pytest.mark.asyncio
async def test_(monocle_trace_asserter: TraceAssertion):
    # 1. Run the agent under instrumentation
    await monocle_trace_asserter.run_agent_async(
        , "", ""
    )

    # 2. Assertions — one chained call per logical check.
    # Chaining narrows context: called_agent(A).called_tool(T) means "tool T inside agent A".
    monocle_trace_asserter \
        .called_agent("") \
        .called_tool("") \
        .contains_input("") \
        .contains_output("")

    # 3. Optional: end-to-end output / performance
    monocle_trace_asserter.contains_output("")
    monocle_trace_asserter.under_token_limit(1_000_000)
```

For a sync agent, drop `@pytest.mark.asyncio`, drop `await`, and call `run_agent(...)`.

## Fluent API cheat sheet

Assertion methods return `self`. Span selectors (`called_tool`, `called_agent`, and their `does_not_*` counterparts) **narrow the span context** for every subsequent assertion in the chain — that's how you say "this tool, called inside this agent, received this input."

Every assertion accepts an optional `message="..."` kwarg for a clearer failure string.

### Run / load

| Call | Use |
|---|---|
| `await asserter.run_agent_async(agent, agent_type, *args, session_id=None)` | Drive an async agent |
| `asserter.run_agent(agent, agent_type, *args)` | Drive a sync agent |
| `asserter.load_spans(JSONSpanLoader.from_json("traces/x.json"))` | Offline: load pre-recorded spans instead of running |

### Span selectors (narrow context)

| Call | Use |
|---|---|
| `.called_tool(tool_name, agent_name=None)` | Assert tool was called; optionally only inside that agent |
| `.does_not_call_tool(tool_name, agent_name=None)` | Assert tool was NOT called |
| `.called_agent(agent_name)` | Assert agent was invoked |
| `.does_not_call_agent(agent_name)` | Assert agent was NOT invoked |

### Input / output content

| Exact | Substring |
|---|---|
| `.has_input(s)` / `.has_output(s)` | `.contains_input(s)` / `.contains_output(s)` |
| `.has_any_input(*s)` / `.has_any_output(*s)` | `.contains_any_input(*s)` / `.contains_any_output(*s)` |
| `.does_not_have_input(s)` / `.does_not_have_output(s)` | `.does_not_contain_input(s)` / `.does_not_contain_output(s)` |
| `.does_not_have_any_input(*s)` / `.does_not_have_any_output(*s)` | `.does_not_contain_any_input(*s)` / `.does_not_contain_any_output(*s)` |

`with_comparer("similarity" \| "default" \| "bert_score" \| "metric" \| "token_match")` overrides equality for `has_*` (not `contains_*`).

### Performance

| Call | Use |
|---|---|
| `.under_token_limit(N)` | Total tokens across selected spans `) to find the registered names. ADK/CrewAI/LlamaIndex agents and tools have explicit string names that must match exactly.
- **Don't omit `@pytest.mark.asyncio` on async tests.** Without it, the coroutine is never awaited and the test silently passes.
- **Don't chain unrelated checks on the narrowed context.** After `called_tool(T)`, an `under_token_limit` applies only to that tool's spans. If you want a workflow-wide token limit, call it on a fresh `monocle_trace_asserter` line (not chained).
- **`contains_*` ignores the configured comparer.** It always does substring matching. Use `has_*` + `with_comparer("similarity")` for semantic matching.
- **Don't write declarative `TestCase` lists in a fluent test.** Those are the `@monocle_testcase` style — a different skill. The fluent file should contain only `async def test_...` functions using `monocle_trace_asserter`.
- **The pytest plugin is auto-loaded.** If running outside the repo where `monocle_test_tools` is installed, the user's `conftest.py` must register `pytest_plugins = ["monocle_test_tools.pytest_plugin"]` — the repo root and `test_tools/tests/integration/conftest.py` already do this, so new tests in that directory need no extra setup.

## After generating

1. Print the file path you wrote to.
2. Show the user the suggested command to run it, e.g. `pytest test_tools/tests/integration/test__fluent.py -v`.
3. Do NOT run the test automatically unless the user asked — most agent runs hit live LLM APIs and cost money.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [monocle2ai](https://github.com/monocle2ai)
- **Source:** [monocle2ai/skills](https://github.com/monocle2ai/skills)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-monocle2ai-skills-monocle-test-gen
- Seller: https://agentstack.voostack.com/s/monocle2ai
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.