Ai Engineer

AI Engineer

When to Use

• Building chatbots, copilots, or retrieval-augmented generation systems
• Designing multi-step agent workflows with tool use
• Integrating OpenAI, Anthropic, or local models into products
• Setting up RAG pipelines (chunk, embed, index, retrieve, rerank, generate)
• Building evaluation harnesses and regression suites for LLM features
• Optimizing cost/latency through model routing, caching, or context strategy
• Planning safe deployment of generative features (canary, kill switch, monitoring)

When NOT to Use

• Academic literature synthesis or research methodology → ai-researcher
• Organizational AI policy, regulation, or risk tiering → ai-risk-governance
• Adversarial safety testing and jailbreak campaigns → ai-redteam
• Prompt-only tuning without system architecture changes → prompt-engineer
• Enterprise-wide non-AI system integration ADRs → senior-system-architecture
• Token/cost improvement program planning and roadmap → ai-token-improvement-plan-engineer
• Commercial/enterprise AI solution architecture → applied-ai-architect-commercial-enterprise
• Skills portfolio governance and batch validation → ai-skill-manager
• Agent prompts, golden evals, judge rubrics → prompt-engineer-agent-prompts-evals

Related skills

| Need | Skill | |---|---| | Prompt templates and agent message design | prompt-engineer | | Offline experiments, statistics, classical ML | data-scientist | | Papers, benchmarks, research methodology | ai-researcher | | Policies, model cards, governance | ai-risk-governance | | Red-team and jailbreak campaigns | ai-redteam | | Persistent memory design | ai-memory-developer | | Context window and token budgeting | ai-context-engineer | | Token cost improvement plan and roadmap | ai-token-improvement-plan-engineer | | AI production ops and release governance | ai-lead-ops | | Cross-system boundaries and platform ADRs | senior-system-architecture | | Commercial/enterprise AI architecture | applied-ai-architect-commercial-enterprise | | Agent skills catalog and validation | ai-skill-manager | | Safeguard serving stack and policy runtime | ml-infrastructure-engineer-safeguards | | Safety model R&D and benchmark design | ml-research-engineer-safeguards |

Core Workflows

1. Solution shaping

1. Define user job, success metric, and failure modes
2. Decide: single LLM call vs RAG vs multi-step agent
3. Choose model tier (quality vs cost vs latency)
4. Identify data sources, PII boundaries, and retention
5. Plan human-in-the-loop for high-risk actions

See references/solution_patterns.md for RAG vs fine-tune vs agent decision tree.

2. RAG pipeline

ingest → chunk → embed → index → retrieve → rerank → generate → cite

Checklist:

• [ ] Chunk size tuned on eval set
• [ ] Metadata filters for tenancy/ACL
• [ ] Hybrid search if keyword matters
• [ ] Ground answers with citations; refuse when context insufficient
• [ ] Refresh index on source updates

See references/rag_pipeline.md for chunking, eval metrics, and freshness.

3. Agents and tools

• Tools: narrow schemas, idempotent where possible, timeouts
• Loop: plan → act → observe → stop condition
• Cap iterations and token budget
• Log tool calls for audit; redact secrets in traces

See references/agents_tools.md for ReAct patterns and failure handling.

4. Evaluation before launch

| Layer | Measure | |---|---| | Retrieval | Recall@k, MRR on golden questions | | Generation | Faithfulness, answer relevance (LLM-judge + human sample) | | Safety | Refusal rate on policy violations | | Ops | p95 latency, cost per session |

Ship only when regression suite passes on CI for golden set.

See references/evaluation_ops.md for datasets, CI eval, and monitoring.

5. Production operations

• Version prompts and models; canary new versions
• Monitor drift, error rate, tool failures, spend
• Kill switch for model or feature flag
• Incident runbook for toxic output or data leak

See references/evaluation_ops.md for production monitoring.

When to load references

• Architecture choices → references/solution_patterns.md
• RAG implementation → references/rag_pipeline.md
• Agents and tools → references/agents_tools.md
• Eval and production → references/evaluation_ops.md

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: daemon-blockint-tech
• Source: daemon-blockint-tech/Agentic-Enteprises-Skill
• License: MIT

Install and usage instructions live in the source repository linked above.