Comfy Cozy

Patent Pending  |  MIT License  |  Patent Details

Comfy Cozy

Talk to ComfyUI like a colleague. It talks back.

You describe what you want in plain English. The agent loads workflows, swaps models, tweaks parameters, installs missing nodes, runs generations, analyzes outputs, and learns what works for you -- all without you touching JSON or hunting through menus. Workflow edits don't wait for permission -- it makes the change, reports what it did, and every change is undoable. The one thing it always asks first: installing node packs or downloading models, because that pulls code and files onto your machine.

graph LR
    You([You]) -->|"make it dreamier"| Agent[Comfy Cozy]
    Agent -->|loads, patches, runs| ComfyUI[ComfyUI]
    ComfyUI -->|image| Agent
    Agent -->|"Done. Lowered CFG to 5,switched to DPM++ 2M Karras.Here's your render."| You

    classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a
    classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a
    class You,ComfyUI orange
    class Agent yellow

> Session 1 is a capable tool. > Session 100 is a capable tool that knows your style.

> TL;DR > - Plain-English co-pilot for ComfyUI. You describe the change; the agent loads workflows, swaps models, patches parameters, runs generations, evaluates output. > - 131 MCP tools across 5 LLM providers — Claude, GPT-4o, Gemini, Ollama, and NVIDIA Nemotron. Swap providers (or models) with one env var or a flag. > - Every mutation is a reversible delta layer (LIVRPS). Full undo stack. Nothing destructive lands without your say-so. > - Experience persists. Session 1 ships with built-in knowledge. After ~30 runs the agent starts biasing toward what's actually worked for you. Every run appends one fsync'd line — never a rewrite, never silent loss. > - The edit loop is fast. Validate → fix → re-validate dropped from ~7 s to ~0.5 s (measured): node schemas are fetched class-scoped and cached, so a re-validate after a fix costs ~1 ms. Interrupted model downloads resume from the byte they died at, with live progress. Long jobs get real per-tool time budgets — a 15-minute video render is no longer killed at 2 minutes. > - Built for the VFX floor. Linear EXR renders flow into the vision loop (ACEScg→sRGB display transform; data passes are refused, not misjudged). Every saved workflow gets a workflow.lock sidecar — model SHA-256s, node-pack commits, ComfyUI version — and validate warns when anything drifted since the lock. COMFYUI_ENDPOINTS pools multiple workers with per-host circuit breakers, health-checked failover, and job affinity. > - Ships three ways: inside Claude Code/Desktop (MCP), standalone CLI, native ComfyUI sidebar. Pick one.

---

Demo

Demo 1 — overview

[](https://vimeo.com/1193607778)

Demo 2 — walkthrough

[](https://vimeo.com/1193622319)

See It In Action

| You say | What happens | |---------|-------------| | "Load my portrait workflow and make it dreamier" | Loads the file, lowers CFG, switches sampler, saves with full undo | | "I want to use Flux" | Searches CivitAI + HuggingFace, downloads the model on your go-ahead, wires it into your workflow | | "Repair this workflow" | Finds missing nodes, shows you the packs it needs, installs them once you approve, fixes connections, migrates deprecated nodes | | "Run this with 30 steps" | Patches the workflow, validates it, queues it to ComfyUI, shows progress | | "Analyze this output" | Uses Vision AI to diagnose issues and suggest parameter changes | | "What model should I use for anime?" | Searches CivitAI + HuggingFace + your local models, recommends the best fit | | "Optimize this for speed" | Profiles GPU usage, checks TensorRT eligibility, applies optimizations | | "Repair and run this" | Finds missing nodes, asks once to install the packs, validates, executes |

---

Sponsor This Project

Comfy Cozy is production software. 4,600+ tests (all mocked, runnable in minutes) cover the 131 MCP tools that drive the workflow lifecycle end-to-end. CI runs the full advertised matrix — Python 3.10–3.13 on Ubuntu and Windows — with the USD stage layer actually installed and tested, not skipped. Five LLM providers — Anthropic, OpenAI, Gemini, Ollama, and NVIDIA Nemotron — sit behind a single abstraction with parity across all five. The [CHANGELOG](CHANGELOG.md) tracks active hardening and new work.

If Comfy Cozy saves you time inside ComfyUI, sponsorship is the most direct way to keep it moving.

Sponsorship funds:

• Continued development of the MCP agent layer
• Priority response on sponsor-filed issues
• New intelligence-layer tools — vision evaluators, provisioning, planning

A separate Pro tier with additional offerings is planned. Details when it's ready, not before.

Become a sponsor →  ·  [Acknowledgments](SPONSORS.md)

---

Get Running

flowchart LR
    A["Clone"] -->|30 sec| B["Install"]
    B -->|10 sec| C["Paste key"]
    C -->|done| D(["agent run"])

    classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a
    classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a
    class D orange
    class A,B,C yellow

Three prerequisites, four copy-paste steps. Under 2 minutes start to finish.

| | What you need | Where to get it | Time | |---|------|-----------------|------| | 1 | Python 3.11+ | python.org/downloads | already have it? skip | | 2 | ComfyUI running | github.com/comfyanonymous/ComfyUI | already have it? skip | | 3 | One LLM backend | API key (Anthropic / OpenAI / Google / NVIDIA) OR Ollama (free, local, no key) | 1 min to grab a key |

Already have all three? Copy-paste these four blocks. That's it.

1. Clone

git clone https://github.com/JosephOIbrahim/Comfy-Cozy.git
cd Comfy-Cozy

2. Install

pip install -e .

One command. No build step. No Docker. No conda. Just pip.

Want the test suite too? (optional, click to expand)

pip install -e ".[dev]"           # + 4,600+ passing tests
pip install -e ".[dev,stage]"     # + USD stage subsystem (~200MB, most users skip)

3. API key

cp .env.example .env

Open .env in any text editor, paste your key on the first line:

ANTHROPIC_API_KEY=sk-ant-your-key-here

Using OpenAI, Gemini, or Ollama instead? (click to expand)

Pick one block, paste into .env:

# --- OpenAI ---
LLM_PROVIDER=openai
OPENAI_API_KEY=sk-your-key-here
# first time only: pip install openai

# --- Gemini ---
LLM_PROVIDER=gemini
GEMINI_API_KEY=your-key-here
# first time only: pip install google-genai

# --- Ollama (fully local, free, no key) ---
LLM_PROVIDER=ollama
AGENT_MODEL=llama3.1
# first time only: ollama pull llama3.1

ComfyUI installed somewhere non-default? (click to expand)

Add one more line to .env:

COMFYUI_DATABASE=C:/path/to/your/ComfyUI

Step 4 of 4 -- Go

agent run

Type what you want. Type quit when you're done.

---

Connect the Sidebar to ComfyUI

The agent also lives inside ComfyUI as a native sidebar panel. To enable it, create two symlinks from ComfyUI's custom_nodes/ folder to Comfy-Cozy:

Windows (run as Administrator):

cd C:\path\to\ComfyUI\custom_nodes
mklink /D comfy-cozy-panel C:\path\to\Comfy-Cozy\panel
mklink /D comfy-cozy-ui C:\path\to\Comfy-Cozy\ui

Linux / macOS:

cd /path/to/ComfyUI/custom_nodes
ln -s /path/to/Comfy-Cozy/panel comfy-cozy-panel
ln -s /path/to/Comfy-Cozy/ui comfy-cozy-ui

Restart ComfyUI. The Comfy Cozy chat panel appears in the left sidebar.

graph LR
    CN["ComfyUI/custom_nodes/"] --> P["comfy-cozy-panel/ (symlink)"]
    CN --> U["comfy-cozy-ui/ (symlink)"]
    P -->|"canvas sync (headless)"| Panel["panel/__init__.py"]
    U -->|"sidebar + chat"| UI["ui/__init__.py"]

    classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a
    classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a
    class CN,U orange
    class P,sync,Panel,UI yellow

Both symlinks are required:

• comfy-cozy-panel -- Canvas sync bridge (runs headlessly -- keeps the agent in sync with your live graph)
• comfy-cozy-ui -- The visible sidebar: chat window, quick actions, status

---

Optional: the Agent Bridge node pack (live canvas + render timing)

Want the agent to load workflows straight onto your canvas, see the edits you make by hand, and tell you exactly which node ate your render time? Install the one extra node pack. It's optional -- everything else works without it -- but it unlocks three things ComfyUI's stock API can't give the agent.

graph LR
    Agent[Comfy Cozy] -->|"push_workflow_to_canvas"| Gate{"Origin + Bearergate"}
    Gate -->|"POST /agent/push_workflow"| Canvas["Your ComfyUI canvas(every open tab reloads)"]
    Canvas -->|"you hand-edit a node"| Gate
    Gate -->|"POST /agent/canvas_changed"| Buf["canvas buffer"]
    Buf -->|"get_canvas_state"| Agent
    Render["Your render finishes"] -->|"per-node timing"| Prof["GET /agent/exec_profile/{id}"]
    Prof -->|"get_execution_profile"| Agent

    classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a
    classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a
    class Canvas,Render orange
    class Agent,Gate,Buf,Prof yellow

> The mutating bridge routes are Origin-gated (a browser fetch can't attach a custom header, so same-origin is the auth layer) and Bearer-gated for non-browser callers when MCP_AUTH_TOKEN is set — so a cross-origin page can't hijack your canvas.

What you get:

• "Load my portrait workflow onto the canvas" -- it appears in your browser, no file juggling
• "What did I just change?" -- the agent reads your hand edits back off the canvas
• "Why was that render slow?" -- real per-node timing, so optimization stops being guesswork

Install (one symlink, then restart):

Windows (run as Administrator):

cd C:\path\to\ComfyUI\custom_nodes
mklink /D comfy_agent_bridge C:\path\to\Comfy-Cozy\node_pack\comfy_agent_bridge

Linux / macOS:

cd /path/to/ComfyUI/custom_nodes
ln -s /path/to/Comfy-Cozy/node_pack/comfy_agent_bridge comfy_agent_bridge

Restart ComfyUI. You'll see comfy_agent_bridge: routes registered in the log -- that's it. If a bridge feature ever returns a "node pack not installed" message, it means ComfyUI needs a restart to pick the pack up.

> Under the hood: these three routes are covered by a live contract test suite (tests/integration/test_bridge_routes_integration.py) that boots against a real ComfyUI and asserts each route answers correctly -- so "installed but silently broken" can't slip through. Run it with python -m pytest tests/integration/test_bridge_routes_integration.py -v while ComfyUI is running.

---

Pick Your LLM

Comfy Cozy is provider-agnostic. Same 131 tools, same streaming, same vision analysis -- swap one env var, or swap models mid-session.

Anthropic (default)

# .env
ANTHROPIC_API_KEY=sk-ant-your-key-here

# Optional overrides (defaults shown):
#   AGENT_MODEL=claude-opus-4-7                  -- main loop
#   FAST_MODEL=claude-haiku-4-5-20251001         -- low-stakes triage
#   VISION_MODEL=claude-opus-4-7                 -- analyze/compare images
#   THINKING_BUDGET=4000                         -- agent reasoning budget (tokens)
#   VISION_THINKING_BUDGET=2000                  -- vision reasoning budget

# Run
agent run

Ships as the default with Opus 4.7 + extended thinking + three-tier prompt caching. The agent runs on Opus 4.7 with a 4K reasoning budget; vision analysis (analyze_image, compare_outputs, suggest_improvements) runs the same model with its own budget. Set FAST_MODEL if you want to route triage / classification tools to Haiku 4.5.

OpenAI

# Install the SDK (one time)
pip install openai

# .env
LLM_PROVIDER=openai
OPENAI_API_KEY=sk-your-key-here
AGENT_MODEL=gpt-4o           # or gpt-4o-mini for faster/cheaper

# Run
agent run

Full tool-use support with streaming. Works with any OpenAI-compatible endpoint.

Google Gemini

# Install the SDK (one time)
pip install google-genai

# .env
LLM_PROVIDER=gemini
GEMINI_API_KEY=your-key-here
AGENT_MODEL=gemini-2.5-flash  # or gemini-2.5-pro

# Run
agent run

Function declarations mapped automatically. Supports Gemini's thinking mode.

Ollama (fully local, no API key)

# Install Ollama: https://ollama.com
# Pull a model
ollama pull llama3.1

# .env
LLM_PROVIDER=ollama
AGENT_MODEL=llama3.1          # or any model you've pulled

# Run (no API key needed)
agent run

Uses Ollama's OpenAI-compatible endpoint at localhost:11434. Override with OLLAMA_BASE_URL if running remotely. No data leaves your machine.

NVIDIA Nemotron (NIM cloud / OpenRouter / self-hosted)

# Install the SDK (one time)
pip install openai

# .env
LLM_PROVIDER=nvidia
NVIDIA_BASE_URL=https://integrate.api.nvidia.com/v1   # or OpenRouter / self-hosted vLLM
NVIDIA_API_KEY=nvapi-your-key-here
AGENT_MODEL=nvidia/nemotron-3-super-120b-a12b         # super (default) / ultra / nano

# Run
agent run --model nemotron

Endpoint-agnostic, OpenAI-compatible provider for NVIDIA's Nemotron-3 reasoning models — grab a key + free credits at build.nvidia.com. The same provider serves NVIDIA NIM cloud, OpenRouter, and a self-hosted vLLM/SGLang endpoint; the model id picks the backend. Nemotron streams ` reasoning, which the provider strips from both the visible output **and** the replayed history by default (reasoning is off unless you set NVIDIAEMITREASONING=true). Vision stays on a multimodal provider (VISIONPROVIDER, default anthropic), so swapping the loop to a text-only Nemotron never breaks analyze_image`.

Swap models on the fly

Beyond the LLM_PROVIDER env var, switch the reasoning model per launch or mid-conversation — no restart, atomic (rolls back on a bad key), and it reaches the live loop immediately:

agent run --model nemotron        # alias -> nvidia/nemotron-3-super-120b-a12b
agent run --model claude          # back to the default
agent run --provider openai --model gpt-4o

In conversation, the swap_model and list_models_available tools do the same thing.

Architecture

All five providers share the same abstraction layer (agent/llm/):

graph LR
    Agent[Agent Loop131 tools] --> LLM{LLM_PROVIDER}
    LLM -->|anthropic| A["ClaudeStreaming + Cache"]
    LLM -->|openai| B["GPT-4oTool Calls"]
    LLM -->|gemini| C["GeminiFunction Decl."]
    LLM -->|ollama| D["OllamaLocal + Private"]
    LLM -->|nvidia| E["NemotronNIM / OpenRouter"]

    classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a
    classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a
    class B,D orange
    class Agent,A,C,E,LLM yellow

Common types (TextBlock, ToolUseBlock, LLMResponse), unified error hierarchy, provider-specific format conversion handled internally. Switch providers with one env var -- no code changes. All 5 providers have dedicated test suites (180+ tests) plus a parameterized conformance suite that verifies protocol compliance across providers. Every stream() and create() call is instrumented with llm_call_total and llm_call_duration_seconds metrics (per-provider labels).

Reasoning + caching (Opus 4.7)

The Anthropic path uses two Opus-4.7-specific features the other providers ignore:

1. Extended thinking. Every agent turn ships `thinking={"type": "enabled",

"budgettokens": THINKINGBUDGET}. The streamed ThinkingBlocks include a cryptographic signature`; we capture it and replay it on the next tur

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: JosephOIbrahim
• Source: JosephOIbrahim/Comfy-Cozy
• License: MIT

Install and usage instructions live in the source repository linked above.