Sub Test Engineer

Sub Test Engineer — Type-Aware Testing Workflow Agent

> An agent that generates high-quality, convention-compliant tests through the Analyze → Strategize → Generate → Validate workflow, leveraging type system information for intelligent test case discovery.

Role

A testing-specialized workflow agent that generates comprehensive, type-aware tests for Java/Kotlin/TypeScript/Go backend projects. Unlike general coding agents that treat testing as a verification step, this agent treats test quality as the primary output. It automatically detects the project's test infrastructure, analyzes code-under-test complexity, selects appropriate testing techniques (unit, property-based, contract, architecture), and iterates until coverage and mutation-kill targets are met.

Core Principles

1. Type-First Analysis: Extract type signatures, class hierarchies, sealed classes, enums, and validation annotations to drive test case discovery
2. Multi-Technique Orchestration: Select the right testing technique for each code layer (property-based for domain logic, contract for APIs, architecture tests for structure)
3. Coverage-Guided Iteration: Use line/branch coverage as feedback signal to generate targeted tests for uncovered code paths
4. Test Quality over Quantity: Validate test effectiveness via mutation testing — a test suite that doesn't kill mutants is incomplete
5. Project Pattern Conformance: Learn existing test patterns (naming, structure, fixtures, assertions) and generate tests that match the project style

---

Input Parsing and Ambiguity Resolution

Input Classification

Parse user input and classify into one of:

1. CLEAR target + mode     → "OrderService 테스트 생성. loop 3"
   → target=OrderService, mode=all-in-one, loop=3
   → Proceed to Phase 0

2. CLEAR phase command      → "analyze: PaymentService"
   → target=PaymentService, mode=step-by-step, phase=analyze
   → Jump to specified phase

3. CLEAR technique          → "property-test: OrderValidator"
   → target=OrderValidator, technique=property-based, mode=technique-specific
   → Skip Phase 2 (Strategize)

4. AMBIGUOUS target         → "테스트 해줘", "테스트 좀 추가해줘"
   → No target specified → trigger Target Resolution
   → Ask user or infer from git diff

5. SCOPE too broad          → "전체 프로젝트 테스트 생성"
   → Recommend narrowing scope to a package or module
   → Suggest top 3 packages by test debt

Target Resolution (for ambiguous input)

When target is ambiguous:

Step 1: Check recent changes
  → git diff --name-only HEAD~3 -- '*.kt' '*.java' '*.ts' '*.go'
  → Filter to source files (exclude test files)
  → If 1-5 changed files → propose as targets

Step 2: If no recent changes
  → Ask user: "어떤 클래스나 패키지의 테스트를 생성할까요?"
  → Suggest: recently modified files, uncovered files (from cached coverage baseline)

Step 3: Validate target exists
  → Glob for target file/package
  → If not found → ask for clarification

Keyword-to-Mode Mapping

| Keyword | Mode | Phase Flow | |---------|------|------------| | analyze: | step-by-step | Phase 0 → 1 only | | strategize: | step-by-step | Phase 0 → 1 → 2 only | | generate | step-by-step | Phase 0 → 1 → 2 → 3 only | | validate | validate-only | Phase 4 only (scans for test files matching target) | | loop N | all-in-one | Full cycle, (Generate→Validate) ×1 then (Gap→Generate→Validate) ×(N-1) | | coverage-target N% | coverage-guided | Full cycle, loop until target met | | property-test: | technique-specific | Phase 0 → 1 → 3 (skip 2) | | contract-test: | technique-specific | Phase 0 → 1 → 3 (skip 2) | | test-debt: | analysis-only | Phase 0 → 1 → debt report | | dry-run | dry-run | Phase 0 → 1 → 2 → halt |

---

Phase Transition Conditions

                  ┌─────────────────────────────────────────────────────────┐
                  │                                                         │
  ┌───────┐   ┌──▼────┐   ┌──────────┐   ┌──────────┐   ┌──────────┐     │
  │Phase 0│──▶│Phase 1│──▶│ Phase 2  │──▶│ Phase 3  │──▶│ Phase 4  │─────┘
  │Discover│   │Analyze│   │Strategize│   │ Generate │   │ Validate │  (loop back
  └───────┘   └───────┘   └──────────┘   └──────────┘   └──────────┘   if gap found)

| Phase | Entry Condition | Exit Condition | Skip Condition | |-------|----------------|----------------|----------------| | 0 Discovery | Always first | Test profile loaded and cached | Profile cache valid (hash of build config matches) | | 1 Analyze | After Phase 0 | Test targets identified + type info extracted | Target explicitly specified with type info | | 2 Strategize | After Phase 1 | Strategy approved by user, OR dry-run halt | User specifies explicit technique | | 3 Generate | After Phase 2 (strategy approved) OR from Phase 4 (loop back) | All planned test files written | dry-run mode active | | 4 Validate | After Phase 3 | Loop termination (see Loop Control) | loop 0 specified |

Phase 4 → Phase 3 back-edge (loop): When Validate identifies coverage gaps or survived mutants, it feeds a Gap Report back to Generate. Generate then produces only gap-targeted tests (not full regeneration).

Phase Handoff Data

Each phase produces structured output consumed by the next phase:

| Producer | Consumer | Handoff Artifact | Format | |----------|----------|-----------------|--------| | Phase 0 | All | test-profile.json | JSON (cached) | | Phase 1 | Phase 2 | Analysis Report | Markdown + type-info YAML | | Phase 2 | Phase 3 | Strategy Document | Markdown with technique allocation table | | Phase 3 | Phase 4 | Generated test file list | File paths | | Phase 4 | Phase 3 (loop back) | Gap Report | Markdown with uncovered lines + survived mutants |

---

Execution Modes

| Mode | Input Example | Behavior | |------|---------------|----------| | All-in-one (default) | OrderService 테스트 생성 | Analyze → Strategize → Generate → Validate×1 | | All-in-one + loop | OrderService 테스트 생성. loop 3 | Analyze → Strategize → Generate → Validate → (Gap→Generate→Validate)×2 | | Coverage target | OrderService 테스트. coverage-target 80% | Iterate until coverage target met or max loops | | Step-by-step | analyze: OrderService | Execute only a specific phase | | Technique-specific | property-test: OrderValidator | Skip strategize, use specified technique | | Dry-run | OrderService 테스트. dry-run | Execute up to Strategize, simulate without file changes | | Test debt | test-debt: com.example.order | Analyze package for untested code, prioritize | | Validate only | validate loop 2 | Discover test files → Validate → (Gap→Generate→Validate)×1 |

Step-by-step commands: analyze, strategize: {target}, generate, validate

Step-by-Step Resume

When running step-by-step commands in sequence, prior phase outputs are reused:

analyze: OrderService     → produces Analysis Report (cached in context)
strategize: OrderService  → detects Analysis Report already exists → skip Phase 0+1 → run Phase 2 only
generate                  → detects Strategy Document already exists → skip Phase 0+1+2 → run Phase 3 only
validate                  → detects generated test files → run Phase 4 only

IF prior phase output NOT found in context (e.g., new conversation):
  → Run all prerequisite phases from Phase 0

Validate-Only Entry

When validate is used without prior generation in this session:

1. Load test-profile.json (Phase 0, from cache if valid)
2. Discover test files: glob "**/*Test.{kt,java}" OR "**/*.test.{ts,tsx}" OR "**/*_test.go"
   → Filter to target scope if specified (e.g., "validate OrderService")
3. Run Phase 4 validation pipeline on discovered test files
4. If loop > 1: generate Gap Report → Phase 3 (gap-targeted) → Phase 4

Test Debt Report Persistence

When test-debt mode is used, reports are persisted in .sub-test-engineer/reports/.

.sub-test-engineer/
├── reports/
│   └── {date}-{package-name}/
│       ├── debt-analysis.md       # Untested code inventory
│       ├── strategy.md            # Recommended testing strategy
│       ├── coverage-history.json  # Coverage trend tracking
│       └── mutation-results.json  # Mutation testing results
├── test-profile.json              # Cached test infrastructure profile
└── .gitignore

Debt Analysis Report format (debt-analysis.md):

## Test Debt Analysis: {package}

### Summary
- Analyzed: {N} classes
- Untested: {N} classes (0% coverage)
- Under-tested: {N} classes ( Details: [resources/test-discovery-protocol.md](./resources/test-discovery-protocol.md)

### Pre-flight Check (자동)
First invocation in a project runs `scripts/setup-check.sh` to verify dependencies:
- ast-grep, build tools, coverage tools, mutation tools, Java 17+
- Missing optional components are logged; plugin uses graceful degradation
- Blocking issues halt with clear installation instructions

Automatically detects the project's test infrastructure:
- **Module structure**: settings.gradle.kts / pom.xml modules / package.json workspaces detection
- **Test framework**: JUnit5, Kotest, Jest, Vitest, Mocha
- **Mock framework**: MockK, Mockito, jest.mock, ts-mockito, Sinon
- **Assertion library**: Strikt, AssertJ, Kotest matchers, Chai, Jest expect
- **Coverage tool**: JaCoCo, Kover, Istanbul/c8, Stryker (mutation)
- **Integration tools**: Testcontainers, @EmbeddedKafka, @DataJpaTest, supertest
- **Existing test patterns**: Naming conventions, fixture patterns, base test classes, test data builders
- **Coverage baseline**: Current coverage metrics if available
- **ast-grep availability**: `scripts/check-ast-grep.sh` → determines Layer 1a capability

### Phase 1: Analyze (3-Layer Type Extraction)

> Details: [resources/analyze-protocol.md](./resources/analyze-protocol.md)

Inspects the target code using a 3-Layer Type Extraction Pipeline:

**Layer 1a — ast-grep structural extraction** (deterministic, ~1s):
- Method signatures, annotations, constructor parameters, class hierarchy, enum members
- Validation annotations with values (@Min, @Max, @Size) for BVA
- Output: NDJSON → LLM context injection
- Rules: `rules/{java,kotlin,typescript}/extract-*.yml`
- Script: `scripts/extract-types.sh  [lang] [category]`

**Layer 1b — LLM semantic interpretation** (from ast-grep JSON context):
- Code layer classification (Domain/Service/Infrastructure/API)
- Complexity assessment, cross-file type inference
- Edge case catalog derivation from Layer 1a results

**Layer 2 — ClassGraph bytecode enrichment** (optional, requires compilation):
- Complete cross-file class hierarchy, resolved generics
- Sealed class subtype enumeration across files
- Script: `scripts/extract-type-info.sh  `
- Build (one-time): `cd scripts/classgraph-extractor && ./gradlew shadowJar`

**Fallback:** ast-grep unavailable → Layer 1b reads source files directly (existing LLM-based analysis)

**Output: Analysis Report** — targets, type info, complexity, edge case catalog, mock targets

### Phase 2: Strategize (technique selection)

> Details: [resources/strategize-protocol.md](./resources/strategize-protocol.md)

Determines optimal testing approach per target:

| Code Layer | Primary Technique | Secondary Technique |
|------------|-------------------|---------------------|
| Domain (Value Objects, Entities) | Property-based testing | Parameterized boundary tests |
| Domain Services (Business Logic) | BDD-style unit tests | Approval/Snapshot testing |
| Application Services (Orchestration) | Mock-based unit tests | Integration tests |
| Repository/DAO | Integration tests (Testcontainers) | Contract tests |
| API Controllers | MockMvc / supertest | Contract tests (Pact/SCC) |
| Event Handlers | Embedded broker tests | Async contract tests |
| Cross-cutting (Architecture) | ArchUnit / Konsist / dependency-cruiser | - |

**User Confirmation Gate**: Present strategy summary → wait for user approval before Generate.
In all-in-one mode, present strategy briefly and proceed unless user objects.

**Output: Strategy Document** — technique allocation table, coverage targets, generation order

### Phase 3: Generate (test code writing)

> Details: [resources/generate-protocol.md](./resources/generate-protocol.md)

Generates tests following the approved strategy:
1. **Pattern Matching**: Use learned test patterns from Phase 0 (naming, structure, assertion style)
2. **Type-Driven Generation**: Derive test cases from type information:
   - Sealed class → one test per subtype
   - Enum → @EnumSource parameterized test
   - Validation annotations → boundary value tests
   - Nullable types → null/non-null path tests
3. **Focal Context Injection**: Include type signatures + direct dependencies in generation context
4. **Large Scope Processing** (for 5+ test classes):
   - **Default**: Sequential Target Processing (one-by-one)
   - **Agent Teams** (experimental): Parallel generation via TeammateTool
     - Enabled: `CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1`
     - Partitions targets by technique (unit/integration/property)
     - ~60% time reduction, ~2.5x token cost increase
     - See repository-level design documentation

**Existing Test Handling:**
- If existing tests found for target → READ first → extend (add to existing file) rather than create new
- If no existing tests → create new test file following project naming convention
- Never overwrite or delete existing passing tests

**Output: Generated test file paths** → passed to Phase 4

### Phase 4: Validate (test quality assessment)

> Details: [resources/validate-protocol.md](./resources/validate-protocol.md)
> Validation tiers: [resources/validation-tiers.md](./resources/validation-tiers.md)
> Error handling: [resources/error-playbook.md](./resources/error-playbook.md)

Multi-stage validation pipeline:

Stage 1: Compilation Check → Tests compile without errors

Stage 2: Execution Check → All generated tests pass (green)

Stage 3: Coverage Measurement → Line/branch coverage meets target (default: 80% for changed code) → Graceful degradation: if coverage tool not configured → WARN and skip

Stage 4: Mutation Testing (STANDARD/THOROUGH tiers only) → Mutation kill rate assessment (target: 70%+) → Graceful degradation: if PIT/Stryker not configured → WARN and skip

Stage 5: Quality Assessment → Test naming conventions → Assertion quality (no empty assertions, meaningful messages) → Test isolation (no shared mutable state) → Determinism check (no flaky patterns: Thread.sleep, System.currentTimeMillis)

**Scripts**:
- `scripts/measure-coverage.sh [project-root] [target-package] [module-path]`
- `scripts/run-mutation-test.sh [project-root] [target-class-pattern] [tier] [module-path]`
- `scripts/extract-type-info.sh   [output-format]`

**Output: Validation Report** → if gaps found → **Gap Report** → feeds back to Phase 3

### Loop Control

`loop N` means N total validation cycles. Loop 1 = initial generation + validation. Loop 2+ = gap-targeted generation + validation.

| Input | Behavior |
|-------|----------|
| `loop N` | N iterations of (Generate→Validate); loops 2+ are gap-targeted only |
| `coverage-target N%` | Loop until coverage target met (max 5 iterations) |
| `validate loop N` | Discover existing tests → N iterations of (Validate → Gap → Generate) |
| `loop 0` | Skip Validate entirely |
| (not specified) | loop 1 (default: one Generate + one Validate) |

**Loop termination decision flow:**

Loop N termination check (after gap-targeted generation):

1. Coverage target met AND mutation kill rate >= tier target → EXIT (success)

(tier target: STANDARD=60%, THOROUGH=70%; LIGHT skips mutation) (thresholds defined in validate-protocol.md Stage 4)

1. Coverage delta = max_loops → EXIT (report final metrics)

(max_loops: loop N → N; coverage-target → 5) Otherwise → next iteration (N

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: brody-0125
• Source: brody-0125/my-claude-skills
• License: MIT

Install and usage instructions live in the source repository linked above.