Agentic Engineering Handbook

Agentic Engineering Handbook

> The definitive OpenAI, Anthropic, Google, MCP, Harness, Evals, and Production Agent Systems learning roadmap.

[](LICENSE) [](#)

If this repository helps you, consider giving it a ⭐

---

Why This Repository?

The AI industry has entered the Agentic Era. Building production-grade AI systems now requires mastering agents, tool use, MCP, memory, long-running workflows, coding agents, agent harnesses, evals, and safety — but the knowledge is scattered across OpenAI blogs, Anthropic engineering posts, SDK docs, cookbooks, and research papers.

This repository consolidates 175 curated resources into one structured learning roadmap.

The goal: Become a world-class Agentic Engineer.

---

How To Use This Handbook

Pick the path that matches your starting point:

• New to agents: follow the [Learning Roadmap](#learning-roadmap) from Phase 0 to Phase 6. Treat each Read First, Then Read, and Build Exercise as a checklist.
• Already building LLM apps: start at [Phase 2](#phase-2--mcp--tool-ecosystem) or [Phase 3](#phase-3--context-memory--skills), then fill gaps in agent loop, tool calling, evals, and production engineering.
• Trying to build projects: use the phase-level Build Exercise prompts, then branch into [Applied Practice Tracks](#applied-practice-tracks) for coding agents, security, code review, or SRE.
• Looking for references: jump to the [Full Reading Table](#full-reading-table). Read P0 first, use P1 for implementation detail, and keep P2 as optional background.

---

Learning Roadmap

Phase 0 — Agent Loop From Scratch

If you treat Claude Code as a coding CLI, many capabilities can feel like magic: it reads files, runs commands, edits code, delegates work, and stays oriented during complex tasks.

From an engineering perspective, the core is much simpler:

model + tools + one loop.

Understanding that loop makes the rest of the system easier to reason about:

• When the agent should plan first, and when it should act immediately
• Why an explicit todo list reduces drift in longer tasks
• Why subagents improve exploration while protecting the main context
• How skills, MCP, and hooks each add capability around the same core loop

These pages are based on the upstream English Markdown tutorials from shareAI-lab/mini-claude-code, with added Study Notes and inline source code for this handbook.

| Step | Page | Code | |------|------|------| | v0 | [Bash is All You Need](tutorials/agent-loop/v0-bash-is-all-you-need.md) | [v0bashagent.py](tutorials/agent-loop/v0bashagent.py) | | v1 | [Model as Agent](tutorials/agent-loop/v1-model-as-agent.md) | [v1basicagent.py](tutorials/agent-loop/v1basicagent.py) | | v2 | [Structured Planning](tutorials/agent-loop/v2-structured-planning.md) | [v2todoagent.py](tutorials/agent-loop/v2todoagent.py) | | v3 | [Subagent Mechanism](tutorials/agent-loop/v3-subagent-mechanism.md) | [v3subagent.py](tutorials/agent-loop/v3subagent.py) | | v4 | [Skills Mechanism](tutorials/agent-loop/v4-skills-mechanism.md) | [v4skillsagent.py](tutorials/agent-loop/v4skillsagent.py) |

Supporting files are included in the same folder: requirements.txt, .env.example, v0_bash_agent_mini.py, and skills/.

---

Phase 1 — Agent Foundations

> Build shared vocabulary for workflow vs agent, tool loop, handoff, guardrails.

Key Mental Models

Should I build an agent? (4-question checklist from Barry Zhang's talk)

| Question | If No → Workflow | If Yes → Agent | |----------|-----------------|----------------| | Is the task complex enough? | Decision tree is fully mappable | Ambiguous problem space | | Is the task valuable enough? | $1 per run, cost doesn't matter | | Are all core capabilities doable? | Weak links break the chain | Model handles every step well | | Is error cost low & detectable? | High cost + hard to detect → human-in-the-loop | Errors caught by tests/CI |

Think like the agent. Most failures come from designing with a human perspective. Put yourself inside the agent's context window: you only see ~10K–20K tokens (system prompt + tool descriptions + recent observations). Ask: does the agent have enough information to act correctly at each step?

→ Source: How We Build Effective Agents

Read First

| # | Title | Vendor | |---|-------|--------| | 1 | System Prompts | Anthropic | | 2 | Prompt guidance | OpenAI | | 3 | Function Calling | OpenAI | | 4 | Tool use overview | Anthropic | | 5 | Function calling - Gemini API | Google | | 6 | Building effective agents | Anthropic | | 7 | New tools for building agents | OpenAI | | 8 | Agents SDK overview | OpenAI |

Then Read

| Title | Vendor | |-------|--------| | How We Build Effective Agents: Barry Zhang, Anthropic | Anthropic | | Phistory — Claude Code & Codex CLI System Prompt Diff History | Community | | Coding Agents 101: The Art of Actually Getting Things Done | Cognition | | OpenAI Agents SDK examples | OpenAI | | Structured Outputs for Multi-Agent Systems | OpenAI |

Build Exercise

Build a customer service/ticket triage agent: router → specialist → evaluator, with all outputs constrained by structured schemas.

---

Phase 2 — MCP & Tool Ecosystem

> Understand MCP server/client, remote vs local, tool loading, approval, connector boundaries.

Read First

| # | Title | Vendor | |---|-------|--------| | 1 | Introducing the Model Context Protocol | Anthropic | | 2 | MCP and Connectors | OpenAI | | 3 | Building MCP servers for ChatGPT Apps and API integrations | OpenAI |

Then Read

| Title | Vendor | |-------|--------| | Code execution with MCP: Building more efficient agents | Anthropic | | Writing effective tools for AI agents - with AI agents | Anthropic | | Model Context Protocol - Codex | OpenAI | | Build a Remote MCP server | Cloudflare | | Introducing the MCP Registry | MCP | | OpenAI Docs MCP | OpenAI | | Build your ChatGPT UI | OpenAI |

Build Exercise

Build a read-only repo/docs MCP server, then create an eval to verify the agent correctly cites documentation.

---

Phase 3 — Context, Memory & Skills

> Learn to control context window, short/long-term memory, skills/plugins, CLAUDE.md/AGENTS.md.

Read First

| # | Title | Vendor | |---|-------|--------| | 1 | Agent Skills Specification | Agent Skills | | 2 | Effective context engineering for AI agents | Anthropic | | 3 | How the Open Knowledge Format can improve data sharing | Google Cloud | | 4 | How Long Contexts Fail | Drew Breunig | | 5 | Context Rot | Chroma | | 6 | Progressive disclosure | Claude-Mem | | 7 | Equipping agents for the real world with Agent Skills | Anthropic | | 8 | Agent Skills | Anthropic | | 9 | Skills | OpenAI | | 10 | Building Reliable Agents with Memory and Compaction | OpenAI |

Then Read

| Title | Vendor | |-------|--------| | Custom instructions with AGENTS.md - Codex | OpenAI | | Best practices for Claude Code | Anthropic | | Agent Skills - Codex | OpenAI | | Skills in OpenAI API | OpenAI |

Build Exercise

Implement the same task as a Skill/Plugin, then measure accuracy and token cost across three variants: no skill, long prompt, and skill-based.

---

Phase 4 — Harness & Long-Running Agents

> Master agent runtime: event stream, thread, tool execution, state, sandbox, approval, recovery.

Read First

| # | Title | Vendor | |---|-------|--------| | 1 | Unrolling the Codex agent loop | OpenAI | | 2 | Unlocking the Codex harness: how we built the App Server | OpenAI | | 3 | Agent Harness Engineering: A Survey | Academic | | 4 | Effective harnesses for long-running agents | Anthropic | | 5 | Deep Agents | LangChain |

Then Read

| Title | Vendor | |-------|--------| | Deep research | OpenAI | | Open Deep Research | LangChain | | The next evolution of the Agents SDK | OpenAI | | Using PLANS.md for multi-hour problem solving | OpenAI | | Build long-running AI agents that pause, resume, and never lose context with ADK | Google | | Harness design for long-running application development | Anthropic | | Scaling Managed Agents: Decoupling the brain from the hands | Anthropic |

Build Exercise

Build a mini coding harness: plan file, shell tool, apply patch, test gate, event log, and resume capability.

---

Phase 5 — Coding & Workspace Agents

> Compare Codex vs Claude Code product/SDK forms; learn multi-agent, IDE, workspace collaboration.

Read First

| # | Title | Vendor | |---|-------|--------| | 1 | AGENTS.md | Agentic AI Foundation | | 2 | Introducing Codex | OpenAI | | 3 | Best practices for Claude Code | Anthropic | | 4 | How Claude Code works in large codebases | Anthropic | | 5 | Enabling Claude Code to work more autonomously | Anthropic |

Then Read

| Title | Vendor | |-------|--------| | Introducing the Codex app | OpenAI | | Introducing workspace agents in ChatGPT | OpenAI | | Apple's Xcode now supports Claude Agent SDK | Anthropic | | Building Consistent Workflows with Codex CLI & Agents SDK | OpenAI | | Best practices for Claude Code | Anthropic | | The spec is dead, long live the spec! | Ravi on Product | | How Anthropic teams use Claude Code | Anthropic | | Multi-stack Web App Builds | Community |

Build Exercise

Run both OpenAI/Codex and Claude Code style workflows on the same repo: issue → plan → patch → tests → PR summary.

---

Phase 6 — Evals, Safety & Production

> Build pre/post-launch eval loop, trace loop, safety boundaries, permissions, regression monitoring.

Read First

| # | Title | Vendor | |---|-------|--------| | 1 | Demystifying evals for AI agents | Anthropic | | 2 | The six generations of AI agents and how to eval them | Braintrust | | 3 | Agent observability powers agent evaluation | LangChain | | 4 | Agent Evaluation Readiness Checklist | LangChain | | 5 | Build an Agent Improvement Loop with Traces, Evals, and Codex | OpenAI | | 6 | Macro Evals for Agentic Systems | OpenAI | | 7 | Testing Agent Skills Systematically with Evals | OpenAI |

Then Read

| Title | Vendor | |-------|--------| | How we build evals for Deep Agents | LangChain | | Deep Research Bench | FutureSearch | | How to Evaluate Tool-Calling Agents | Arize | | AI agent evaluation: How to test, debug, and improve agents in production | Arize | | A Survey on Agent-as-a-Judge | Academic | | Running Codex safely at OpenAI | OpenAI | | How we contain Claude across products | Anthropic | | Evals API Use-case - MCP Evaluation | OpenAI | | Measuring AI agent autonomy in practice | Anthropic |

Build Exercise

Build a smoke/macro eval suite for your agent: task success rate, tool misuse, prompt injection resistance, latency, cost, and

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: keyuchen21
• Source: keyuchen21/agentic-engineering-handbook
• License: MIT

Install and usage instructions live in the source repository linked above.