TDPilot

████████╗██████╗ ██████╗ ██╗██╗      ██████╗ ████████╗
╚══██╔══╝██╔══██╗██╔══██╗██║██║     ██╔═══██╗╚══██╔══╝
   ██║   ██║  ██║██████╔╝██║██║     ██║   ██║   ██║
   ██║   ██║  ██║██╔═══╝ ██║██║     ██║   ██║   ██║
   ██║   ██████╔╝██║     ██║███████╗╚██████╔╝   ██║
   ╚═╝   ╚═════╝ ╚═╝     ╚═╝╚══════╝ ╚═════╝    ╚═╝

TDPilot Runtime v2.0.3

[](https://github.com/dreamrec/TDPilot/actions/workflows/ci.yml) [](https://www.npmjs.com/package/tdpilot) [](https://www.npmjs.com/package/tdpilot) [](./LICENSE) [](./pyproject.toml) [](./docs/API_REFERENCE.md) [](https://derivative.ca)

TDPilot Runtime is an MCP server for TouchDesigner. It lets an AI agent inspect, build, wire, optimize, and stabilize live TD networks with real tool calls — and now remember what works. The v2 brain ships with a 656-card reviewed operator atlas with zero-concept backlog, so concept-to-node planning can stay grounded in Official Derivative docs instead of vague recipes.

#tdpilot #touchdesigner #mcp #livepatch #audioreactive #realtime

Install — Claude Code plugin (recommended)

Easiest: paste these two slash commands into any Claude Code session.

/plugin marketplace add dreamrec/TDPilot
/plugin install tdpilot@dreamrec-TDPilot

That installs all 112 MCP tools, 8 skills (tdpilot-core, tdpilot-production, popx-touchdesigner, plus 5 brain skills), 4 brain agents, 2 slash commands (/td-check, /td-snapshot), and the TD-side .tox component — one command, no Python setup required.

Shell one-liner alternative:

curl -fsSL https://raw.githubusercontent.com/dreamrec/TDPilot/main/scripts/install_claude_plugin.sh | bash

Or via npx:

npx tdpilot plugin-install

TouchDesigner side (once, after install)

Drag ~/.claude/plugins/cache/dreamrec-TDPilot/tdpilot//td_component/tdpilot.tox into your TD /local container. Or paste the auto-setup Python block from [docs/INSTALL_CLAUDE_PLUGIN.md](docs/INSTALLCLAUDEPLUGIN.md) into the Textport (auto-detects the latest installed version).

Using Claude Desktop instead of Claude Code? See [docs/INSTALL_CLAUDE_PLUGIN.md](docs/INSTALLCLAUDEPLUGIN.md) — the two flows shouldn't be mixed on one machine.

Documentation

• Install (Claude Code plugin): docs/INSTALL_CLAUDE_PLUGIN.md
• Getting started: docs/GETTING_STARTED.md
• User guide: docs/USER_GUIDE.md
• Memory guide: docs/MEMORY_GUIDE.md
• Production manual: docs/MANUAL.md
• API reference: docs/API_REFERENCE.md
• Concept-to-node master plan: docs/TDPILOT_CONCEPT_TO_NODE_MASTER_PLAN.md
• Effectiveness roadmap: docs/TDPILOT_EFFECTIVENESS_ROADMAP.md
• Security model: docs/SECURITY.md
• Troubleshooting: docs/TROUBLESHOOTING.md
• MCP 1.1 surface (historical, v1.1 — see API reference above for the current 112-tool surface): docs/MCP_1_1_SURFACE.md
• Release notes: CHANGELOG.md

What This Is

• A practical control layer between AI agents and TouchDesigner.
• A structured toolset for scene edits, diagnostics, event monitoring, and recovery.
• A workflow-oriented MCP built for iterative patch development, not one-shot guessing.
• A correctness-first visual programming brain that plans TD concepts before it mutates networks.
• A technique memory system that learns from your projects and builds a reusable library.
• A 656-card reviewed operator atlas with zero-concept backlog, covering CHOP, COMP, DAT, MAT, POP, SOP, and TOP with Official Derivative docs, key_concepts, key_params, and gotchas.
• A 50+ case concept-to-node golden eval corpus gated by scripts/eval_brain_golden.py, covering compiler-backed patterns, assembly macros, generated-code paths, device-source prompts, and stable output conventions.
• 112-tool runtime surface with focus + locations, hint injection, component notes, knowledge corpus, vision diagnostics, TD 2025 native inspection, official recommendations, job resources, memory, optimizer, safety, POPx inspection, project lifecycle control, custom parameter authoring, typed patch sessions, BrainPlan transactions, optional cockpit rendering, agent activity log, one-tool self-update, sync-status checks, and sync diagnosis.

Packaged Add-ons

• Reviewed operator atlas — The core local add-on for translating abstract ideas into real TD operators. Agents can use the 656-card reviewed operator atlas, Official Derivative source URLs, params, concepts, and gotchas to choose smaller, safer operator chains.
• Brain skills and agents — Codex and Claude Code both ship explorer, builder, validator, recovery, and release workflows, so the same inspect -> plan -> execute -> validate discipline works in either client.
• Hooks and release guards — Local deterministic checks catch plugin mirror drift, personal path leaks, stale .tox state, and unsafe release handoffs.
• Optional local knowledge packs — POPX and future specialty packs stay local and user-owned; they extend planning context without adding hosted-service dependencies.

Start Here: Core Workflow

You don't need all 112 tools. Start with these and expand as needed:

| Step | Tools | What You're Doing | |------|-------|-------------------| | Plan | td_brain_plan | Ground the user's visual intent in real TD operators, the reviewed atlas, Official Derivative docs, constraints, hints, memory, and live state | | Execute | td_brain_execute, td_transaction_apply | Apply only a valid BrainPlan or PatchPlan with preflight, snapshots, rollback, and validation | | Inspect | td_get_info, td_get_nodes, td_get_params, td_get_errors | Understand current state before touching anything | | Check memory | td_memory_recall | See if a reusable technique already exists | | Build | td_create_node, td_connect_nodes, td_set_params | Make changes in small, reversible steps | | Verify | td_get_errors, td_cooking_info, td_screenshot | Prove the change worked | | Protect | td_snapshot_scene, td_restore_snapshot | Save milestones, roll back if needed | | Remember | td_memory_learn, td_memory_save | Save successful patterns for reuse |

The loop: Inspect -> Plan -> Execute transactionally -> Validate -> Snapshot or Roll back -> Learn only if validated.

Everything else (vision, streaming, optimization, planning, TD2025 inspection) builds on top of this core.

What's New In 2.0.3

TDPilot v2.0.3 keeps the correctness-first visual programming brain and tightens the practical edges around installation truth, wrapper behavior, and token-efficient vision capture:

• td_sync_status — one-call truth for server version, live TD component version, .tox freshness, plugin cache versions, npm/GitHub latest, and public GitHub description drift.
• td_sync_diagnose — strict local/live drift report for package versions, plugin cache versions, running endpoint, live component version, and shared-secret fingerprints without printing secrets.
• Vision capture hardening — metadata-only frame capture now asks the TD component to omit image bytes at the source; confirmed capture still returns base64 when explicitly requested.
• Wrapper hardening — focused tests now cover the thinnest public wrappers around events, info, system diagnostics, knowledge store, vision, streaming, notes, safety, state, data, and optimizer behavior.
• Release gates — CI and local checks now catch README/skill-frontmatter version drift and run the cheap brain/plugin audits earlier.

The v2.0.0 brain release turned the mature 1.6 tool surface into a correctness-first visual programming brain. It added four public tools and a safer default workflow:

• td_brain_plan — read-only planner that converts visual intent into VisualTaskSpec -> ConceptGraph -> BrainPlan -> PatchPlan. It grounds plans in live TD state, operator cards, hints, memory, data-domain compatibility, missing facts, and concept validators. Blocked plans return questions instead of guessing.
• td_brain_execute — mutates only from a valid BrainPlan, applies transaction defaults, validates the resulting network, rolls back on apply or validation failure, exports a trace, and learns only from validated outcomes when requested.
• td_transaction_apply — safe low-level executor for PatchPlan or BrainPlan, with preflight, max-op limits, snapshots, dry-run support, dependency ordering, rollback flags, and validation profiles.
• td_cockpit_render — read-only MCP Apps cockpit payload for concept graph, transaction, validation, rollback, and trace summaries. The open MCP core stays local-first and has no hosted LLM dependency.
• Brain validators and atlas coverage — concept profiles now cover feedback, audio-reactive, POP, GLSL, render pipeline, panel UI, control rigs, and generic TOP/CHOP/SOP/DAT chains. The 656-card reviewed operator atlas now has zero-concept backlog across CHOP, COMP, DAT, MAT, POP, SOP, and TOP, and scripts/audit_brain_atlas.py gates it.
• Concept-to-node evals — the 50+ case concept-to-node golden eval corpus is checked by scripts/eval_brain_golden.py, with coverage for compiler-backed candidate graphs, pattern composition, assembly macros, generated-code diagnostics, device-source constraints, and stable/debug outputs.
• Client packaging — Codex and Claude plugin surfaces include brain skills, brain agents, deterministic hooks, MCP prompts, live cached resources, schema snapshots, and release audits.

Recent v1.6 groundwork remains in place: read journals, agent activity log, self-update, hint routing, TD 2025 knowledge cards, and the one-button .tox installer.

• v1.6.16 — Agent-observability + self-update. Every tool response now carries a _read_journal hint (call_count, result_unchanged, first_seen_at, last_seen_at) so Claude can see across MCP request boundaries which reads have moved and which haven't — no more wasted token cycles re-fetching the same view of td_get_nodes. A 200-entry server-side activity ring buffer mirrors to an in-TD Table DAT (/local/mcp_server/activity_log) so users can wire agent activity into their visuals. New td_self_update MCP tool hits the GitHub releases API and (optionally) writes the latest tdpilot.tox to all three install paths (repo, Claude Code plugin cache, ~/.tdpilot/) with md5 sync reporting — closing the seven-layer staleness saga documented in user memory.

• v1.6.14 — MCP-server-side auth fix: TDClient now resolves the shared secret fresh on every request (env → ~/.tdpilot/.tdpilot.env → constructor fallback, with 5s cache) and retries once on a 401 after invalidating the cache. Symmetric with the v1.6.13 TD-side fallback — closes the asymmetric half that was still 401ing real sessions when bootstrap_auth ran late and a stale module-level secret got baked into the cached httpx.AsyncClient headers. Also ships the TD 2025.32820 release card (Math Mix / Math Combine POPs, EXR compression, GLSL MAT TDProjTextureLod/TDProjTextureSize, Blackmagic SDK 16, CUDA 12.9.1, NDI 6.3.1) and a CI freshness gate (scripts/check_release_notes_freshness.py) that fails when seed cards trail docs.derivative.ca/Release_Notes by more than one build.
• v1.6.13 — Permanent TD-side auth-race fix: _current_shared_secret() in td_component/mcp_webserver_callbacks.py now has a file fallback that reads ~/.tdpilot/.tdpilot.env when os.environ is empty. Stops the 401 cascade when TD is opened directly (Dock icon, double-click .toe) and inherits an empty env, then npx tdpilot later writes the secret to the file.
• v1.6.12 — Critical: npm wrapper at npm/run.js no longer breaks Claude Desktop's stdio MCP transport. Pre-1.6.12 it emitted progress messages on stdout via console.log, but Claude Desktop listens on stdout for JSON-RPC, so every non-JSON line triggered Unexpected token 'T'… parse errors. All wrapper output now goes to stderr.
• v1.6.11 — DeepSeek-v4 backports: structured recovery_hints forwarded through td_tool_batch sub-results so the agent gets actionable error context per failed call; byte-stable JSON in preference_store and snapshot_manager (sorted keys + stable separators) so file diffs reflect actual state changes, not serializer entropy; hint-corpus growth across error_recovery, panel_ui, pop, popx, recording, and custom_parameters topics.
• v1.6.10 — "Pin this project to disk .tox" pulse + Body status row in the installer panel. Closes the v1.6.6 auto-update gap: pre-1.6.10 the _save_toe_with_externaltox mechanism only protected ~/.tdpilot/tdpilot_default.toe; user-created .toe files in arbitrary locations had a frozen embedded COMP body that wouldn't auto-update. One click → externaltox attached → save with saveExternalToxs=False → reopen .toe → fresh content → automatic updates from then on.
• v1.6.2 — Hint schema v2 with when.surface routing. Optional per-hint surface list (create_node, set_params, exec, errors, plan, preview, query, inspect, screenshot) means hints fire only when the matching surface is in scope. Each tool's auto-injection passes its natural surface automatically; explicit td_get_hints callers can narrow with surface=....
• v1.6.1 — Hint corpus expansion: 11 → 20 packs, 41 → 73 hints. New packs added for audio_reactive, custom_parameters, error_recovery, extensions, feedback, glsl, macros, panel_ui, pop, popx, recording, render_pipeline topics; op-type-keyed packs for audiofileinCHOP, extensionDAT, feedbackTOP, geometryCOMP, glslMAT, glslTOP, moviefileoutTOP, panelCOMP.

The v1.6.3 – v1.6.9 line was a six-release sequence around panel rendering + restricted-mode bypass mechanics — captured in docs/TD_INTRICACIES_AND_PATTERNS.md (the dev-machine reference doc that came out of that postmortem). Full per-version detail: see [CHANGELOG.md](CHANGELOG.md).

What's New In 1.6.0

Cockpit ergonomics release — the agent now feels TD-native without anyone building a new in-TD UI panel. Tool count 99 → 103. Pure host-side; no .tox rebuild required.

• td_get_focus — returns current network pane, selection, and project meta. Eliminates the cold-start "what path are you working in?" tax that the agent paid before every patch.
• td_locations(action=save|list|go|delete|rename) — per-project named network locations stored at ~/.tdpilot/locations/.json. Survives session restarts and follows the .toe across machines.
• td_get_hints(topic, op_type, intent, error_text) — concise, source-cited rules for a topic, op type, or intent. Pure host-side orchestrator over the YAML hint corpus at src/td_mcp/hints/packs/. Ships with 7 packs (17 hints): feedback, glsl, renderpipeline, audioreactive, extensions, feedbackTOP, geometryCOMP.
• Auto-hint injection on 6 high-risk tools — td_create_node, td_set_params, td_exec_python, td_get_errors, td_plan_patch, td_patch_preview automatically attach a hints block when an injection rule matches (creating a feedbackTOP, assigning a string to a reference param, etc.). All 6 also accept include_hints=False for forced opt-in.
• td_component_notes(action=…) — per-COMP markdown notes addressable by path. Default external storage (no .toe bloat); embed=True mirrors into a hidden Text DAT inside the COMP for portability. Pairs with the new td_get_node_detail(include_notes=True).
• td_search_nodes scopes — scopes=["dat_text", "param_exprs"] extends the existing search to DAT text contents and parameter expressions. Backward-compatible with the legacy search_type argument.

The deferred-features list (library, AI adapters, VST, native TD shell, cloud, etc.) lives in local roadmap-future.md per the §13 adoption rules — refusing parity work is a feature.

What's New In 1.5.6

One-button installer release. The shipped tdpilot.tox is now a self-installing componen

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: dreamrec
• Source: dreamrec/TDPilot
• License: MIT
• Homepage: https://github.com/dreamrec/TDPilot/releases/latest

Install and usage instructions live in the source repository linked above.