Ai Harness Setup

AI Harness Setup

Use this skill to bootstrap a repository toward a durable agent-driven workflow: deep inspection first, then APM as the dependency and skill delivery layer, spec-driven development for change management (OpenSpec by default, or the repo's existing system), deterministic checks matched to the detected stack, AI skills for review and delivery, a docs tree that explains the whole system, and AI IDE configuration for every IDE the team uses.

Quick Start

1. Inspect first. Search the repo's manifests, lockfiles, CI configs, framework configs, and existing docs to classify the full stack before changing anything. Record: languages, package managers, type checkers, test runners, linters, CI system, monorepo layout, and existing tooling.
2. Read the matching stack guide based on detection results:

• references/stacks/javascript-typescript/stack.md
• references/stacks/python/stack.md
• references/stacks/java/stack.md

1. If the repo uses a major framework, also read the matching framework guide:

• references/stacks/javascript-typescript/framework-react.md
• references/stacks/javascript-typescript/framework-angular.md
• references/stacks/java/framework-spring-boot.md

1. Ask whether the team wants GitHub Agentic Workflows enabled in this repository.
2. Ask whether deterministic checks should be enforced or advisory before configuring any checks. Carry that answer through all downstream local scripts, CI wiring, and git hooks.
3. If GitHub Agentic Workflows are enabled, read references/github-agentic-workflows.md before installing AI workflow tooling.
4. Read references/ai-tooling.md before initializing APM or installing package-backed skills.
5. Read references/mcp-servers.md before adding extra MCP servers to apm.yml.
6. Read references/openspec.md for change management setup.
7. Read references/deterministic-checks-core.md and the matching per-stack deterministic-scans.md together -- they form one phase.
8. Read references/dependabot.md when adding Dependabot configuration.
9. Read references/docs-bootstrap.md when creating the docs section that explains the workflow.
10. Read references/opencode.md when OpenCode or ocx support is requested.
11. Verify is mandatory. After all setup steps finish, run the Verify step in Workflow Step 10 to audit the changeset against the harness instructions, fix every gap found, and produce a verify report. Never skip this step.

For non-trivial setup work, use subagents to split independent exploration or authoring tasks across stack, tooling, validation, and documentation domains, then merge the results in one final coordinating pass.

Workflow

1. Inspect the Repo and Detect the Full Stack

Run this step before making any changes. The results drive every decision in later steps.

• Search manifests, lockfiles, CI configs, framework configs, dependency declarations, and existing docs.
• Do not stop at top-level markers. Search deeply for framework configs, nested package files, and CI pipeline definitions.
• Ask early whether the team wants GitHub Agentic Workflows enabled and whether deterministic checks should be enforced or advisory so the setup path can either include or skip that layer intentionally and wire checks correctly.
• Detect and record:
• Languages and their versions
• Package managers (npm, pnpm, bun, yarn, uv, poetry, pip, Maven, Gradle)
• Type checkers already in use (TypeScript, mypy, basedpyright, pyright, pytype)
• Linters already in use (ESLint, ruff, checkstyle, spotbugs)
• Test runners (vitest, jest, pytest, junit, playwright)
• Security scanners already present (bandit, Semgrep, npm audit, pip-audit, OWASP dependency-check)
• CI system (Jenkins, GitHub Actions, GitLab CI) and its config files
• Monorepo layout (uv workspaces, pnpm workspaces, Nx, Turborepo, Gradle multi-module, Maven multi-module)
• Frameworks (React, Next.js, Angular, FastAPI, LangGraph, Spring Boot, etc.)
• Existing docs, AGENTS.md, CONTRIBUTING.md
• Existing skills, spec-driven development artifacts (OpenSpec openspec/, SpecKit specify/ or specs/, BMAD, or other structured planning systems)
• AI IDE configs already present: Cursor (.cursor/, .cursorrules), GitHub Copilot (.github/copilot-instructions.md, .github/copilot-*), Windsurf (.windsurf/, .windsurfrules), Claude Code (.claude/, CLAUDE.md), OpenCode (opencode.jsonc, .opencode/)
• MCP surfaces and automation entry points
• Record all findings. This detection output drives steps 2-10.

Monorepo detection: If the repo is a monorepo (workspace files, multiple packages, shared build tooling), note the workspace layout, which packages have their own tooling configs, and whether checks should run at workspace level vs. per-package level. Adapt all later steps accordingly.

• Look for frameworks and tools such as React, Next.js, Vite, Electron, LangChain, LangGraph, CopilotKit, FastAPI, Hono, Axum, Tauri, Playwright, Docker, Jenkins, and GitHub Actions.
• When a framework is detected, load its framework-specific reference for use during the deterministic checks and skills phases.

1b. Preflight: Check .gitignore for Path Conflicts

Before creating any directories or files, verify that the paths this workflow will generate are not ignored by .gitignore:

git check-ignore docs/ openspec/ .opencode/ .opencode/skills/ .github/skills/ .github/prompts/ .github/instructions/ .github/agents/ .github/hooks/ .claude/ .claude/skills/

If any of these paths are listed in .gitignore, warn the user and resolve the conflict before proceeding. Content created under ignored paths will not appear in git status and will silently fail to commit.

APM deployed files should be committed. Files that APM places in .github/ (prompts, skills, instructions, agents, hooks) and .claude/ are intended to be version-controlled so that every contributor and Copilot on github.com gets agent context without running apm install. Only apm_modules/ should be gitignored (APM handles this automatically).

To catch drift between apm.yml and deployed files, use a CI drift check rather than gitignore. See references/ai-tooling.md for the drift check pattern.

2. Initialize APM

• Treat APM as the dependency and delivery foundation for skills, prompts, and related agent packages.
• Detect whether the repo already has apm.yml, apm.lock.yaml, apm_modules/, or APM-managed .github/skills/, .claude/skills/, .cursor/skills/, or .opencode/skills/ directories.
• If apm.yml does not exist, initialize it before installing skills:
• run apm --version
• run apm init --yes from the repo root
• If apm.yml already exists, inspect it before adding dependencies, then use apm install to sync the current dependency graph when needed.
• Verify apm_modules/ is in .gitignore (apm init should add it automatically; if not, add it manually).
• Use imperative apm install commands for skill installation so the manifest and lockfile stay aligned with the actual installed set.
• Use references/ai-tooling.md for initialization, install order, discovery strategy, and verification.

3. Set Up Spec-Driven Development

Ensure the repo has a spec-driven development system for managing non-trivial changes. Use the detection results from Step 1 to determine what already exists.

If a spec-driven development system already exists (SpecKit via specify/ or specs/, BMAD, or any structured planning system with real artifacts):

• Respect it. DO NOT replace it with OpenSpec.
• Document the existing system in the docs tree (Step 7), CONTRIBUTING.md, and AGENTS.md.
• Wire its workflow commands into the AI IDE configs (Step 8) if it has a CLI or command surface.
• If the system has no CLI or command surface, document the manual workflow contributors should follow.

If no spec-driven development system exists:

• Install OpenSpec as the default.
• Use references/openspec.md for the install sequence, openspec init, command surfacing, and verification flow.
• Confirm openspec/changes/ and openspec/specs/ exist after initialization (with .gitkeep if empty).
• Add OpenSpec workflow commands to the AI IDE command surfaces (Step 8).

4. Add Deterministic Checks

Combine the language-agnostic baseline with stack-specific checks in a single pass, informed by the detection results from step 1.

Before configuring any deterministic checks, ask: "Do you want deterministic checks enforced or advisory?" Apply that one answer uniformly to all deterministic checks, CI steps, and git hooks that this workflow wires in.

Language-agnostic baseline (use references/deterministic-checks-core.md):

• dependency or vulnerability audit
• SAST or Semgrep-style scanning
• tests
• build or packaging verification
• CI parity with local commands

Stack-specific checks (use references/deterministic-scans.md as the overview, then the matching per-stack file):

• Read the core baseline first, then the stack-specific guide, then act once.
• Use the package manager, type checker, linter, and test runner detected in step 1. Do not override existing tooling with defaults.
• If the repo already has a tool fulfilling a check category (e.g., basedpyright for type checking), use it instead of recommending the default.
• Keep per-language commands and setup instructions in separate reference files instead of bloating the main workflow.

Monorepo considerations:

• Determine whether checks run at the workspace root or per-package.
• Use workspace-aware commands when available (e.g., pnpm nx affected -t lint, uv run with workspace targeting).
• Document selective testing strategy if the repo has one (e.g., change detection scripts).

CI wiring:

• Match every deterministic check added locally with a corresponding CI stage or step.
• Detect the CI system from step 1 and wire checks into the appropriate config format.
• For Jenkins repos, add checks as stages in the Jenkinsfile or as calls to existing shared library functions. See references/ci-wiring.md for patterns.
• For GitHub Actions repos, add checks as workflow steps.
• Reuse existing CI scripts if the repo already encodes the right invariant.

5. Add Dependabot Configuration

Ensure the repo has a Dependabot configuration for automated dependency updates based on the ecosystems detected in Step 1.

• Use references/dependabot.md for the guard-rails, ecosystem mapping, defaults, and monorepo handling.
• If .github/dependabot.yml or .github/dependabot.yaml already exists, leave it untouched.
• If neither exists, create .github/dependabot.yml with entries for each detected ecosystem.

6. Install AI Tooling and Skills

• Ask whether GitHub Agentic Workflows should be enabled before changing AI tooling.
• If the answer is yes, use references/github-agentic-workflows.md as the source of truth for gh aw installation, workflow import, generated files, and post-install token guidance.
• If the answer is no, skip gh aw setup entirely.

Use a package-first install flow. The curated APM packages are the default delivery path for skills and any MCP servers they already bundle.

• Start with apm --version.
• Audit repo-local skill directories and existing APM dependencies before installing duplicates.
• Initialize APM first if the repo does not already have apm.yml.
• Resolve the latest published package tag at install time. Use git ls-remote --tags https://github.com/cisco-open/ai-harness-toolkit "-v*", sort the returned tags by semver, and install the newest matching tag. Do not hardcode package versions in this skill.
• Use protocol fallback in every package install command:

apm install --allow-protocol-fallback cisco-open/ai-harness-toolkit/packages/#-v

• Choose the default package from the detection results in step 1:

| Detected stack | Install | | --- | --- | | JavaScript/TypeScript, no framework | stack-javascript-typescript | | Vue, Svelte, or other generic web UI | stack-frontend | | React or Next.js | stack-react | | Angular | stack-angular | | Python managed with uv | stack-python-uv | | Java with Spring Boot | stack-spring-boot |

• Every stack-* package already pulls core transitively. Do not install core separately when a stack package matches.
• If no stack package matches (for example Go, Rust, plain Java, or another unpackaged backend), install core explicitly and then add only the extra dynamic skills that the detected stack still needs.
• Use references/ai-tooling.md for the install mechanics, what-to-commit rules, and dynamic fallback workflow.

Dynamic fallback and add-ons:

• Use the matching package as the default baseline when one exists, then add only the extra dynamic skills that cover real repo-specific gaps.
• Browse repo-local and public curated sources first for Cisco-curated skills that match the detected stack.
• Use npx skills find only as a search mechanism when the needed skill is not already available through repo-local or public curated sources.
• If no package matches (for example FastAPI, Django, plain Java, Go, Rust, LangGraph, or CopilotKit), keep that part of the stack dynamic.

6b. Add MCP Servers

Add MCP servers to apm.yml based on the detected tech stack. MCP servers give agents runtime access to external tools and services.

• Read references/mcp-servers.md for the package-backed defaults, discovery workflow, and APM dependency format.
• Inspect the repo for existing MCP declarations in opencode.jsonc, .opencode/, Cursor, Claude, and similar config files, then migrate those definitions into apm.yml so APM becomes the single source of truth.
• Treat package-provided MCP servers as already handled by the corresponding package install:
• stack-frontend, stack-react, and stack-angular provide Chrome DevTools.
• Treat those package-provided MCP servers as the baseline, not the ceiling. Add extra MCP servers when the detected stack still justifies them.
• Use apm mcp search only for extra servers not covered by a package, such as Postgres, Playwright, Sentry, Kubernetes, Terraform, or a public design-system server. Only add servers that provide clear value for the repo's actual workflow.
• All MCP servers are declared in the dependencies.mcp section of apm.yml and installed through apm install.
• Preserve any existing MCP entries in apm.yml and carry forward any repo-local MCP config that should remain supported.

7. Create the Docs Section That Explains the Workflow

Synthesis checkpoint: Before writing docs, consolidate all findings from steps 1-6. The docs must accurately reflect everything that was detected, installed, and configured.

• Verify docs/ is not gitignored. Run git check-ignore docs/ before creating any files. If docs/ is ignored, update .gitignore first -- otherwise the entire docs tree will be invisible to git.
• Research deeply before writing. Read source code, configs, CI pipelines, deployment configs, data models, API definitions, and integration points. Every doc should contain material that could only come from actually reading this repo.
• Use references/docs-bootstrap.md for the full seed structure, research checklist, and authoring order.
• Create architecture docs first (system overview, services, data model, integrations) because they require the deepest research and inform everything else.
• Create dev environment docs with real commands (setup, run, env vars).
• Create validation docs that reference the deterministic checks wired in step 4.
• Create deployment docs if the repo ships to production.
• Write the workflow overview last -- it synthesizes all other docs.
• Create AGENTS.md as a lean routing file that points agents to docs/. Do not dump content into AGENTS.md that belongs in focused docs.
• Update `CO

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: cisco-open
• Source: cisco-open/ai-harness-toolkit
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.