Agent Ready

AgentReady

Transform any repository into a structured, verifiable context layer for AI coding agents.

[](https://opensource.org/licenses/MIT) [](https://www.python.org/downloads/) [](https://github.com/vb-nattamai/agent-ready/releases)

---

Overview

AI coding agents operating on unfamiliar repositories produce unreliable output not because of model capability limitations, but because they lack structured knowledge of the codebase. Without explicit context, agents hallucinate file paths, invent build commands, and make incorrect assumptions about domain logic and project conventions.

AgentReady addresses this at the source. It analyses a repository's structure, source files, CI configuration, and documentation, then generates a set of platform-specific context files that make the codebase legible to any AI agent. It then measures whether those files actually improve agent behaviour using a grounded evaluation framework.

Supported providers: Claude · OpenAI · Gemini · Groq · Mistral · Together · Ollama

---

Table of Contents

• [How It Works](#how-it-works)
• [Generated Artifacts](#generated-artifacts)
• [Skills and Hooks](#skills-and-hooks)
• [Evaluation Framework](#evaluation-framework)
• [Quick Start](#quick-start)
• [Cost and Time](#cost-and-time)
• [Requirements](#requirements)
• [CLI Reference](#cli-reference)
• [GitHub Actions Integration](#github-actions-integration)
• [Model Strategy](#model-strategy)
• [Context Freshness](#context-freshness)
• [PR Review Agent](#pr-review-agent)
• [agent-context.json Structure](#agent-contextjson-structure)
• [Language and Framework Support](#language-and-framework-support)
• [Gitea Support](#gitea-support)
• [Design Principles](#design-principles)
• [Troubleshooting](#troubleshooting)
• [Contributing](#contributing)
• [License](#license)

---

How It Works

AgentReady operates as a five-phase pipeline:

Phase 1: Collect Mechanically reads the file tree, source files, configuration, CI pipelines, README, and build definitions. No LLM involved — pure file I/O.

Phase 2: Analyse The analysis model reads the collected files and extracts domain concepts, entry points, environment variables, module layout, and known operational pitfalls specific to the codebase.

Phase 3: Generate The generation model produces all scaffolding files from scratch based on the analysis output. No templates are filled in. Files are written for each supported agent platform, including the new skills/, hooks/, and .cursorrules artifacts.

Phase 4: Score Computes a 100-point agentic readiness score based on which structured context criteria are satisfied.

Phase 5: Evaluate The evaluation model runs 19 structured questions across five categories against the repository, comparing responses with and without the generated context files. Ground truth is derived from raw source code, not from the generated files, eliminating circularity. Results are written to AGENTIC_EVAL.md and surfaced in the pull request.

The output is a pull request containing all generated files and a quantified eval report.

---

Generated Artifacts

| File | Purpose | |---|---| | AGENTS.md | Operating contract for GitHub Copilot and OpenAI agents — defines safe operations, forbidden operations, and domain glossary | | CLAUDE.md | Automatically loaded by Claude Code at session start — includes module layout, conventions, and critical commands | | .cursorrules | Automatically loaded by Cursor at project open — equivalent to CLAUDE.md for Cursor users | | system_prompt.md | Universal system prompt compatible with any LLM interface | | agent-context.json | Machine-readable repository map with static and dynamic sections | | mcp.json | MCP server configuration for Claude and MCP-compatible clients | | openapi.yaml | Auto-generated OpenAPI 3.1 stub — fill in paths and schemas, validate with Redocly | | memory/schema.md | Agent working memory and state contract | | skills/ | Slash-command skill definitions for repo-specific agent actions (run-tests, build, lint, etc.) | | tools/ | Helper scripts for maintaining the agent context — includes refresh_context.py to re-run AgentReady and keep agent-context.json current as the codebase evolves | | cost_report.json | Token usage and estimated USD cost breakdown per model — covers generation and evaluation phases | | hooks/ | Session-continuity hooks for Claude Code (session-start, pre-tool-call, post-test, pre-commit) | | AGENTIC_EVAL.md | Evaluation report showing baseline and with-context scores per category |

---

Skills and Hooks

AgentReady generates repo-specific skill definitions and session hooks in addition to context files. These go beyond telling the agent what the repo is — they tell it what it can do and how to maintain state across sessions.

Skills

Skills are invocable slash-command definitions placed in the skills/ directory. Each skill is a self-contained instruction set for a specific repo action, grounded in the commands detected during analysis.

Skills generated depend on what the repo contains:

| Detected signal | Skill generated | |---|---| | Test runner detected | skills/run-tests.md | | Build command detected | skills/build.md | | Linter detected | skills/lint.md | | Docker or docker-compose present | skills/start-local.md | | Migration framework detected | skills/run-migrations.md | | CI config present | skills/run-ci.md | | OpenAPI spec present | skills/generate-api-docs.md | | Package manager detected | skills/add-dependency.md |

skills/run-tests.md and skills/build.md are always generated. If exact commands are not determinable from source, the skill file explicitly states what is not known and where to find the information. Commands in skill files come from agent-context.json. Skills never invent commands.

Hooks

Hooks are session continuity definitions placed in the hooks/ directory. They fire at specific points in the Claude Code lifecycle to load current repository state and enforce constraints.

| Hook | When it fires | Always generated | |---|---|---| | hooks/session-start.md | Start of every Claude Code session | Yes | | hooks/pre-tool-call.md | Before any file-writing tool call | Yes | | hooks/post-test.md | After running the test command | When test runner detected | | hooks/pre-commit.md | Before a git commit | When linter detected |

The pre-tool-call hook checks the target path against restricted_write_paths in agent-context.json before any write operation. This enforces the non-destructive constraint at the tool level, not just at the instruction level.

---

Evaluation Framework

Every transformation includes a structured evaluation that measures whether the generated context files produce a measurable improvement in agent response quality.

Methodology

| Parameter | Value | |---|---| | Questions | 19 across 5 categories | | Baseline model | claude-sonnet-4-6 with no context | | Context model | claude-sonnet-4-6 with all generated files | | Judge | claude-haiku-4-5 — 3-panel majority vote (factual, semantic, safety) | | Ground truth source | Raw source code — not the generated context files |

> Baseline and context model are the same to isolate context quality as the > only variable. The judge model differs from the eval model to eliminate > scoring bias.

Observed Results

Results from the hello_world example — a minimal Flask REST API with 4 endpoints, pytest tests, pyproject.toml, and CI. Eval design: sonnet vs sonnet baseline, haiku judge.

| Category | Baseline | With Context | Improvement | |---|---|---|---| | Overall | 1.8 / 10 | 7.1 / 10 | +5.3 pts | | Commands | 2.2 / 10 | 7.7 / 10 | +5.5 pts | | Safety | 3.2 / 10 | 5.1 / 10 | +1.9 pts | | Architecture | 1.2 / 10 | 8.6 / 10 | +7.4 pts | | Domain | 0.0 / 10 | 6.5 / 10 | +6.5 pts | | Adversarial | 1.3 / 10 | 6.9 / 10 | +5.6 pts |

The most consistent signal is architecture, which moves from 1.2/10 to 8.6/10 with a 100% pass rate on the hello_world reference repo. Entry point, language, framework, and directory structure are all correctly identified with context.

Commands also shows strong improvement, moving from 2.2/10 to 7.7/10 with 60% pass rate. Build commands, install commands, and run commands are reliably extracted from Makefile and pyproject.toml.

The safety category reflects a known characteristic of minimal repos: when a codebase has no secrets handling, no restricted paths, and no dangerous operations, the correct answer is "none exist." Context files now correctly state this rather than inventing mechanisms.

Hallucination rate has improved from 79% at initial release to 26% after grounding fixes applied to the analysis and generation pipeline. The remaining failures are concentrated in adversarial questions where the judge penalises mentions of requirements.txt in fallback install commands, even when the generated content correctly identifies pyproject.toml as the authoritative source.

Both baseline and context responses use the same model, so scores reflect the impact of context files alone — not model capability differences. The judge model differs from both to eliminate scoring bias.

The evaluation report produced after each transformation identifies specifically which questions failed and what information was missing, providing an actionable improvement path rather than a single aggregate score.

Language support

The evaluation golden set is currently optimised for Python repositories. Results on Python repos (Flask, FastAPI, Django) will be most accurate.

For non-Python repos, the eval still runs and measures context improvement, but some questions (test framework configuration, packaging) may not map cleanly to the language. Java, Go, and TypeScript golden sets are planned.

Evaluating existing context files

The evaluation framework works on any repository — not just ones AgentReady transformed. If you have written CLAUDE.md or AGENTS.md by hand, run:

agent-ready --target /path/to/repo --eval-only

This scores whatever context files exist and tells you exactly which questions failed and what information was missing. Use it as a benchmark before and after manual edits.

# Evaluate and fail CI if pass rate is below 60%
agent-ready --target /path/to/repo --eval-only --fail-level 0.6

---

Quick Start

The recommended path is the one-click installer available in the AgentReady Actions tab.

1. Navigate to Actions → Install AgentReady to Target Repository
2. Click Run workflow
3. Enter the target repository in owner/repo format and select an LLM provider
4. The installer pushes a trigger workflow to the target repository, opens an issue, and applies the agentic-ready label
5. The transformation runs automatically and opens a pull request with all generated files

> Cost notice: A full transformation including evaluation makes approximately > 100-140 LLM calls. On Anthropic (default), expect $0.80-1.20 for a minimal repo > and $1.50-3.00 for a large multi-service repo. Use --dry-run to preview what > will be generated without making any API calls. Use --eval false to skip the > evaluation phase and reduce cost by approximately 60%.

Review the pull request, fill in the static section of agent-context.json with project-specific details, and merge.

For a complete example of what AgentReady generates, see [examples/hello_world_output/](examples/helloworldoutput/) — a full transformation output against a minimal Python/Flask project, including skills, hooks, and all context files.

---

Cost and Time

How long does a transformation take?

A full transformation on a typical repository takes 3-8 minutes end to end.

| Phase | Approximate time | |---|---| | Analysis (reads codebase) | 30-90 seconds | | Generation (writes all files) | 60-180 seconds | | Evaluation (19 questions) | 60-120 seconds | | Total | 3-8 minutes |

Time scales with repository size. A minimal single-file repo completes in under 3 minutes. A large multi-module repo with many source files may take up to 10 minutes.

How many LLM calls does it make?

A full transformation makes approximately 50-65 LLM calls depending on which artifacts are generated.

| Stage | Calls | |---|---| | Analysis | 1 | | Generation (per artifact) | 1 per file — approximately 8-12 total | | Skill generation | 1 per skill — 2-8 depending on repo | | Hook generation | 1 per hook — 2-4 depending on repo | | Evaluation (19 questions × 2 models) | 38 | | Total | Approximately 50-65 |

Approximate cost per provider

Costs are based on observed runs including the full evaluation phase (38 LLM calls across baseline and context models). A minimal single-file repo costs approximately $1.35. Larger repos with more source files will cost more.

| Provider | Model tier | Approximate cost per run | |---|---|---| | Anthropic (default) | Opus / Sonnet / Haiku | $1.00 - $1.50 | | OpenAI | GPT-4o / GPT-4o-mini | $0.10 - $0.30 | | Google | Gemini 2.5 Pro / Flash | $0.05 - $0.20 | | Groq | Llama 3.3 70B | $0.01 - $0.05 | | Mistral | Large / Small | $0.05 - $0.15 | | Together | Qwen / Llama | $0.02 - $0.08 | | Ollama | Local models | Free (local compute only) |

> These are estimates based on typical repository sizes. Your actual cost depends on the number of source files, their length, and which artifacts are generated. Run with --dry-run first to preview what will be generated without incurring any cost.

How to reduce cost

• Use --provider groq or --provider together for the lowest cost per run.
• Use --eval-only or omit --eval to skip the evaluation step — this removes the 38 evaluation calls and reduces cost by approximately 60%.
• Use --only agents or --only context to regenerate specific artifacts only, rather than running the full pipeline.
• Use Ollama for fully local, zero-cost runs (quality will vary by model).

---

Requirements

• Python 3.9 or higher
• An API key for the chosen LLM provider

# Anthropic (default)
export ANTHROPIC_API_KEY="sk-ant-..."

# OpenAI
export OPENAI_API_KEY="sk-..."

# Google
export GOOGLE_API_KEY="..."

# Groq
export GROQ_API_KEY="..."

# Mistral
export MISTRAL_API_KEY="..."

# Together
export TOGETHER_API_KEY="..."

# Ollama — no key required; requires a running local Ollama instance

---

CLI Reference

# Install (with LLM support)
pip install "git+https://github.com/vb-nattamai/agent-ready.git[ai]"

# Or install from source for development
git clone https://github.com/vb-nattamai/agent-ready.git
cd agent-ready
pip install -e '.[dev]'

| Command | Description | |---|---| | agent-ready --target /path/to/repo --provider anthropic | Full transformation with evaluation | | agent-ready --target /path/to/repo --dry-run | Preview generated files without writing | | agent-ready --target /path/to/repo --only context --force | Regenerate context map only | | agent-ready --target /path/to/repo --only agents | Regenerate agent instruction files only | | agent-ready --target /path/to/repo --eval | Run evaluation after transformation | | agent-ready --target /path/to/repo --eval-only | Evaluate existing context files without running transformation | | agent-ready --target /path/to/repo --eval-only --fail-level 0.6 | Evaluate and fail if pass rate is below 60% | | agent-ready --target /path/to/repo --review-pr 42 | Run PR review agent against PR number 42 | | agent-ready --target /path/to/repo --eval --fail-level 0.8 | Fail if eval pass rate is below 80% — use as a CI gate | | agent-ready --target /path/to/repo --quiet | Suppress output for CI pipelines |

---

GitHub Actions Integration

Trigger Mechanism

The installer pushes a workflow to the target repository that triggers on issue

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: vb-nattamai
• Source: vb-nattamai/agent-ready
• License: MIT
• Homepage: https://github.com/vb-nattamai/agent-ready

Install and usage instructions live in the source repository linked above.