Hierarchical Orchestration Router

Hierarchical Orchestration Router

Overview

Discovery and retrieval find the right tool. Orchestration coordinates them at scale. As tool counts grow from dozens to hundreds and agent counts from one to many, you need infrastructure that routes requests, manages failover, and enforces governance. This skill composes three ideas from the chapter's "Orchestration at Scale" section.

Inversion — one orchestrator, not thousands of tools. Instead of exposing every tool to the agent, expose exactly one: an intelligent orchestrator that handles all complexity. The agent asks for an outcome; the orchestrator decides how to achieve it. This is what lets traditional SaaS answer "why did we lose deals last quarter?" in seconds instead of through menus and reports.

Hierarchical routing. Organizations don't just have more tools — they have multiple MCP servers across departments (Sales, Finance, Operations), each managing hundreds of tools. The router classifies a query into a domain by semantics. High confidence (> 0.8) routes to that domain's orchestrator; low confidence means the query spans domains, so it invokes cross-domain orchestration (chapter Example 6-11). This buys fault isolation (a logistics outage does not stop sales), scalable governance (global vs domain-local policies), and progressive disclosure (users see only capabilities relevant to their context).

Functional clustering for resilience. Within a domain, tools with similar function are clustered for intelligent failover. Baidu's AI Search Paradigm embeds tools by what they DO (DRAFT-refined docs + usage patterns), then K-means++ groups them into functional toolkits. When the primary tool is overloaded, the orchestrator fails over to a functionally-equivalent alternative from the same cluster — a "Search Toolkit" of Baidu AI Search / ArXiv MCP / Perplexity / OpenAI WebSearch — adapting parameters as it fails over. No single point of failure.

When to Use

• Multiple departments/domains each own many tools (or MCP servers)
• Queries arrive that may belong to one domain or span several
• You need failover: when one tool is overloaded, route to an equivalent
• You are turning a legacy multi-tool surface into a single natural-language entry

Phrases that invoke this skill: "route this query", "which domain owns this", "cross-domain", "one orchestrator", "fail over to an equivalent tool", "functional clustering", "hierarchical orchestration".

When NOT to Use

• A single-domain system with a handful of tools — routing overhead exceeds

benefit; use rag-mcp-tool-selection directly.

• As tool retrieval within a domain — this routes to a domain and names the

functional clusters; picking the specific tool inside a domain is rag-mcp-tool-selection / the gateway.

• As a security boundary — per-domain access control (which agent may reach

which domain) is a separate authentication layer; this is coordination.

• When there is no failover set — a domain with one tool per function has no

functionally-equivalent alternative; the skill correctly reports a single point of failure rather than inventing one.

Process

| Step | Input | Action | Output | Verification | |------|-------|--------|--------|--------------| | 1 | Orchestration config JSON (domains + tools) | lib.load_config(path) | Config dict | Has domains (with keywords + tools) and tools (with key_topics) | | 2 | Query + domains | lib.identify_domain(query, domains) | Best domain + margin confidence [0,1] | Single-domain query → confidence near 1.0; spanning query → near 0.5 | | 3 | Query + domains + threshold | lib.route_request(query, domains, threshold=0.8) | {routing: domain\|cross_domain, ...} | confidence ≥ 0.8 → domain; ` — it must report no alternative, not invent a cross-function one.

1. JSON round-trips.

`` python cli.py orchestrate "..." --json | python -c "import json,sys; json.load(sys.stdin)" ``

Security Posture

• Prompt injection. Domain keywords and tool topics are author-controlled;

treat them as untrusted if sourced from external registries. Routing decisions are deterministic given the config — a malicious keyword set could misroute, so the config is a governed artifact.

• Data exfiltration. No network calls in lib.py. The # TODO(production):

seams in identify_domain (embedding classifier) and cluster_tools (K-means++ over embeddings) mark where real ML backends attach.

• Privilege escalation. Cross-domain routing must still honor per-domain

access control — routing a query to Finance does not grant the agent Finance tools; that gate is mcp-gateway-two-meta-tools / IAM. This skill decides WHERE a query goes, not WHETHER the agent may go there.

Composition

• Sits above rag-mcp-tool-selection / mcp-gateway-two-meta-tools: the

router picks the domain; those pick the tool within it.

• Consumes draft-tool-trust-verifier output: functional clustering embeds

tools "by what they do, not what they claim" (DRAFT-refined representations), and the reliability failover ordering is a performance-based trust score.

• Feeds the Enterprise AI OS control plane — the router is the semantic-search
• action-governance step in the chapter's EnterpriseAIOS.handle_agent_request.
• Pairs with federated-context-governance — hierarchical routing needs a

coherent per-domain context to route into.

Source Attribution

Distilled from Agentic GraphRAG (O'Reilly), Chapter 6 — Tool Orchestration, "Orchestration at Scale": "The Intelligent Orchestrator", "Hierarchical Orchestration: When Scale Demands Structure" (Example 6-11), and "Functional Clustering: Resilience Through Redundancy". Named references:

• MCP Gateway hierarchical orchestration — Enterprise Orchestrator → domain →

tools; identify_domain confidence 0.8 routing threshold

• Baidu AI Search Paradigm — functional clustering via K-means++ over

usage-pattern embeddings; Search Toolkit failover example

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: AnthonyAlcaraz
• Source: AnthonyAlcaraz/agentic-graph-rag-skills
• License: MIT
• Homepage: https://www.oreilly.com/library/view/agentic-graphrag/9798341623163/

Install and usage instructions live in the source repository linked above.