DiffuGen

DiffuGen - Advanced Local Image Generator with MCP Integration

Your AI art studio embedded directly in code. Generate, iterate, and perfect visual concepts through this powerful MCP server for Cursor, Windsurf, and other compatible IDEs, utilizing cutting-edge Flux and Stable Diffusion models without disrupting your development process.

> ⭐ New: Now includes OpenAPI server support and OpenWebUI OpenAPI Tools (OWUI Version 0.60.0 Required) integration for seamless image generation and display in chat interfaces! The OpenAPI is seperate from the MCP server and allowss for initigrations into your own projects!

📃 Table of Contents

• [Introduction](#-introduction)
• [Understanding MCP and DiffuGen](#-understanding-mcp-and-diffugen)
• [Features](#-features)
• [System Requirements](#-system-requirements)
• [Installation](#-installation)
• [IDE Setup Instructions](#-ide-setup-instructions)
• [Usage](#-usage)
• [OpenAPI Server Usage](#openapi-server-usage)
• [Default Parameters by Model](#default-parameters-by-model)
• [Asking a LLM to Generate Images](#asking-a-llm-to-generate-images)
• [Parameter Reference](#parameter-reference)
• [Model-Specific Parameter Recommendations](#model-specific-parameter-recommendations)
• [Default Parameter Changes](#default-parameter-changes)
• [Command Line Usage Notes](#command-line-usage-notes)
• [Configuration](#️-configuration)
• [Configuration Approach](#configuration-approach)
• [Environment Variable Overrides](#environment-variable-overrides)
• [Setting IDE-Specific Configurations](#setting-ide-specific-configurations)
• [Key Configuration Elements](#key-configuration-elements)
• [IDE-Specific Options](#ide-specific-options)
• [Customizing Default Parameters](#customizing-default-parameters)
• [Updating Configuration Files](#updating-configuration-files)
• [Advanced Usage](#-advanced-usage)
• [Using the OpenAPI Server](#using-the-openapi-server)
• [License](#-license)
• [Acknowledgments](#-acknowledgments)
• [Contact](#-contact)

🚀 Introduction

DiffuGen is a powerful MCP-based image generation system that brings cutting-edge AI models directly into your development workflow. It seamlessly integrates both Flux models (Flux Schnell, Flux Dev) and Stable Diffusion variants (SDXL, SD3, SD1.5) into a unified interface, allowing you to leverage the unique strengths of each model family without switching tools. With comprehensive parameter control and multi-GPU support, DiffuGen scales from rapid concept sketches on modest hardware to production-quality visuals on high-performance systems.

Built on top of the highly optimized stable-diffusion.cpp implementation, DiffuGen offers exceptional performance even on modest hardware while maintaining high-quality output.

🧠 Understanding MCP and DiffuGen

What is MCP?

MCP (Model Context Protocol) is a protocol that enables LLMs (Large Language Models) to access custom tools and services. In simple terms, an MCP client (like Cursor, Windsurf, Roo Code, or Cline) can make requests to MCP servers to access tools that they provide.

DiffuGen as an MCP Server

DiffuGen functions as an MCP server that provides text-to-image generation capabilities. It implements the MCP protocol to allow compatible IDEs to send generation requests and receive generated images.

The server exposes two main tools:

1. generate_stable_diffusion_image: Generate with Stable Diffusion models
2. generate_flux_image: Generate with Flux models

Technical Architecture

DiffuGen consists of several key components:

• setup-diffugen.sh: The complete install utility and model downloader and manager
• diffugen.py: The core Python script that implements the MCP server functionality and defines the generation tools
• diffugen.sh: A shell script launcher that sets up the environment and launches the Python server
• diffugen.json: Template configuration file for MCP integration with various IDEs (to be copied into IDE's MCP configuration)
• stable-diffusion.cpp: The optimized C++ implementation of Stable Diffusion used for actual image generation

The system works by:

1. Receiving prompt and parameter data from an MCP client
2. Processing the request through the Python server
3. Calling the stable-diffusion.cpp binary with appropriate parameters
4. Saving the generated image to a configured output directory
5. Returning the path and metadata of the generated image to the client

About stable-diffusion.cpp

stable-diffusion.cpp is a highly optimized C++ implementation of the Stable Diffusion algorithm. Compared to the Python reference implementation, it offers:

• Significantly faster inference speed (up to 3-4x faster)
• Lower memory usage (works on GPUs with as little as 4GB VRAM)
• Optimized CUDA kernels for NVIDIA GPUs
• Support for various sampling methods and model formats
• Support for model quantization for better performance
• No Python dependencies for the core generation process

This allows DiffuGen to provide high-quality image generation with exceptional performance, even on modest hardware setups.

✨ Features

• Multiple Model Support: Generate images using various models including Flux Schnell, Flux Dev, SDXL, SD3, and SD1.5
• MCP Integration: Seamlessly integrates with IDEs that support MCP (Cursor, Windsurf, Roo Code, Cline, etc.)
• OpenAPI Server: Additional REST API interface for direct HTTP access to image generation capabilities
• Cross-Platform: Works on Linux, macOS, and Windows (via native or WSL)
• Parameter Control: Fine-tune your generations with controls for:
• Image dimensions (width/height)
• Sampling steps
• CFG scale
• Seed values
• Negative prompts (for SD models only, Flux does not support negative prompts.)
• Sampling methods
• CUDA Acceleration: Utilizes GPU acceleration for faster image generation
• Natural Language Interface: Generate images using simple natural language commands
• Smart Error Recovery: Robust error handling with operation-aware recovery procedures
• User-Friendly Setup: Interactive setup script with improved interrupt handling
• Resource Tracking: Session-aware resource management for efficient cleanup
• Customizable Interface: Support for custom ANSI art logos and visual enhancements

💻 System Requirements

Minimum Requirements:

• CPU: 4-core processor (Intel i5/AMD Ryzen 5 or equivalent)
• RAM: 8GB system memory
• Storage: 5GB free disk space (SSD preferred for faster model loading)
• Python: 3.8 or newer
• GPU: Integrated graphics or entry-level dedicated GPU (optional)
• Network: Broadband connection for model downloads (5+ Mbps)

Recommended Requirements:

• CPU: 8+ core processor (Intel i7/i9 or AMD Ryzen 7/9)
• RAM: 16GB+ system memory
• GPU: NVIDIA GPU with 6GB+ VRAM (RTX 2060 or better for optimal performance)
• Storage: 20GB+ free SSD space
• Python: 3.10 or newer (3.11 offers best performance)
• Network: High-speed connection (20+ Mbps) for efficient model downloads

📥 Installation

Automatic Installation (Recommended)

The easiest way to install DiffuGen is using the provided setup script:

git clone https://github.com/CLOUDWERX-DEV/diffugen.git
cd DiffuGen
chmod +x diffugen.sh
chmod +x setup_diffugen.sh
./setup_diffugen.sh

Follow the interactive prompts to complete the installation.

The setup script will:

• Install necessary dependencies
• Clone and build stable-diffusion.cpp
• Set up a Python virtual environment
• Download selected models (Note: Some models require Clip\VAE Models as well)
• Configure file paths for your system

Manual Installation

If you prefer to install manually, follow these steps:

1. Clone the repositories:

git clone https://github.com/CLOUDWERX-DEV/diffugen.git
cd DiffuGen
git clone --recursive https://github.com/leejet/stable-diffusion.cpp

1. Build stable-diffusion.cpp:

cd stable-diffusion.cpp
mkdir -p build && cd build

With CUDA:

cmake .. -DCMAKE_BUILD_TYPE=Release -DSD_CUDA=ON
make -j$(nproc)
cd ../..

Without CUDA:

cmake .. -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
cd ../..

1. Create and activate a Python virtual environment:

python3 -m venv diffugen_env
source diffugen_env/bin/activate  # On Windows: diffugen_env\Scripts\activate
pip install -r requirements.txt

1. Download required models (structure shown below):

stable-diffusion.cpp/models/
├── ae.sft                           # VAE model
├── clip_l.safetensors               # CLIP model
├── flux/
│   ├── flux1-schnell-q8_0.gguf     # Flux Schnell model (default)
│   └── flux1-dev-q8_0.gguf          # Flux Dev model
├── sd3-medium.safetensors           # SD3 model
├── sdxl-1.0-base.safetensors        # SDXL model
├── sdxl_vae-fp16-fix.safetensors    # SDXL VAE
├── t5xxl_fp16.safetensors           # T5 model
└── v1-5-pruned-emaonly.safetensors  # SD1.5 model

You can download the models from the following sources:

# Create model directories
mkdir -p stable-diffusion.cpp/models/flux

# Flux models
# Flux Schnell - Fast generation model (Q8 Quantized,requires t5xxl, clip-l, vae)
curl -L https://huggingface.co/leejet/FLUX.1-schnell-gguf/resolve/main/flux1-schnell-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-schnell-q8_0.gguf

# Flux Dev - Development model with better quality (Q8 QUantized, requires t5xxl, clip-l, vae)
curl -L https://huggingface.co/leejet/FLUX.1-dev-gguf/resolve/main/flux1-dev-q8_0.gguf -o stable-diffusion.cpp/models/flux/flux1-dev-q8_0.gguf

# Required models for Flux
# T5XXL Text Encoder
curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/t5xxl_fp16.safetensors -o stable-diffusion.cpp/models/t5xxl_fp16.safetensors

# CLIP-L Text Encoder
curl -L https://huggingface.co/Sanami/flux1-dev-gguf/resolve/main/clip_l.safetensors -o stable-diffusion.cpp/models/clip_l.safetensors

# VAE for image decoding
curl -L https://huggingface.co/pretentioushorsefly/flux-models/resolve/main/models/vae/ae.safetensors -o stable-diffusion.cpp/models/ae.sft

# Stable Diffusion models
# SDXL 1.0 Base Model (requires sdxl-vae)
curl -L https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors -o stable-diffusion.cpp/models/sd_xl_base_1.0.safetensors

# SDXL VAE (required for SDXL)
curl -L https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/sdxl_vae-fp16-fix.safetensors -o stable-diffusion.cpp/models/sdxl_vae-fp16-fix.safetensors

# Stable Diffusion 1.5 (standalone)
curl -L https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.safetensors -o stable-diffusion.cpp/models/v1-5-pruned-emaonly.safetensors

# Stable Diffusion 3 Medium (standalone)
curl -L https://huggingface.co/leo009/stable-diffusion-3-medium/resolve/main/sd3_medium_incl_clips_t5xxlfp16.safetensors -o stable-diffusion.cpp/models/sd3_medium_incl_clips_t5xxlfp16.safetensors

Note: Model download may take a long time depending on your internet connection. The SDXL model is approximately 6GB, SD3 is about 13GB, SD1.5 is around 4GB, and Flux models are 8-13GB each.

1. Update file paths in configuration:

Set shell script as Executable

chmod +x diffugen.sh

Configuration Approach: DiffuGen uses a single configuration file (diffugen.json) as the source of truth for all settings. The workflow is:

1. Edit diffugen.json in the DiffuGen root directory with your desired settings
2. Run option 5 in setup_diffugen.sh to automatically update paths in this file
3. Copy the content of diffugen.json to your IDE's MCP configuration file

The file contains all necessary settings:

• File paths (command, SDCPPPATH, modelsdir, outputdir)
• Default model parameters (steps, cfgscale, samplingmethod)
• VRAM usage settings
• Metadata for IDE integration

{
  "mcpServers": {
    "diffugen": {
      "command": "/home/cloudwerxlab/Desktop/Servers/MCP/Tools/DiffuGen/diffugen.sh",
      "args": [],
      "env": {
        "CUDA_VISIBLE_DEVICES": "0",
        "SD_CPP_PATH": "path/to/stable-diffusion.cpp",
        "default_model": "flux-schnell"
      },
      "resources": {
        "models_dir": "path/to/stable-diffusion.cpp/models",
        "output_dir": "path/to/outputs",
        "vram_usage": "adaptive"
      },
      "metadata": {
        "name": "DiffuGen",
        "version": "1.0",
        "description": "Your AI art studio embedded directly in code. Generate, iterate, and perfect visual concepts through this powerful MCP server for Cursor, Windsurf, and other compatible IDEs, utilizing cutting-edge Flux and Stable Diffusion models without disrupting your development process.",
        "author": "CLOUDWERX LAB",
        "homepage": "https://github.com/CLOUDWERX-DEV/diffugen",
        "usage": "Generate images using two primary methods:\n1. Standard generation: 'generate an image of [description]' with optional parameters:\n   - model: Choose from flux-schnell (default), flux-dev, sdxl, sd3, sd15\n   - dimensions: width and height (default: 512x512)\n   - steps: Number of diffusion steps (default: 20, lower for faster generation)\n   - cfg_scale: Guidance scale (default: 7.0, lower for more creative freedom)\n   - seed: For reproducible results (-1 for random)\n   - sampling_method: euler, euler_a (default), heun, dpm2, dpm++2s_a, dpm++2m, dpm++2mv2, lcm\n   - negative_prompt: Specify elements to avoid in the image\n2. Quick Flux generation: 'generate a flux image of [description]' for faster results with fewer steps (default: 4)"
      },
      "cursorOptions": {
        "autoApprove": true,
        "category": "Image Generation",
        "icon": "🖼️",
        "displayName": "DiffuGen"
      },
      "windsurfOptions": {
        "displayName": "DiffuGen",
        "icon": "🖼️",
        "category": "Creative Tools"
      },
      "default_params": {
        "steps": {
          "flux-schnell": 8,
          "flux-dev": 20,
          "sdxl": 20,
          "sd3": 20,
          "sd15": 20
        },
        "cfg_scale": {
          "flux-schnell": 1.0,
          "flux-dev": 1.0,
          "sdxl": 7.0,
          "sd3": 7.0, 
          "sd15": 7.0
        },
        "sampling_method": {
          "flux-schnell": "euler",
          "flux-dev": "euler",
          "sdxl": "euler",
          "sd3": "euler",
          "sd15": "euler"
        }
      }
    }
  }
}

🔧 IDE Setup Instructions

Setting up with Cursor

1. Download and install Cursor
2. Go to Cursor Settings > MCP and click "Add new global MCP server"
3. Copy the contents of your DiffuGen's diffugen.json file and paste it into ~/.cursor/mcp.json
4. Refresh MCP Servers in Settings > MCP
5. Use DiffuGen by opening the AI chat panel (Ctrl+K or Cmd+K) and requesting image generation

Setting up with Windsurf

1. Download and install Windsurf
2. Navigate to Windsurf > Settings > Advanced Settings or Command Palette > Open Windsurf Settings Page
3. Scroll down to the Cascade section and click "Add Server" > "Add custom server +"
4. Copy the contents of your DiffuGen's diffugen.json file and paste into ~/.codeium/windsurf/mcp_config.json
5. Use DiffuGen through the Cascade chat interface

Setting up with Roo Code

1. Download and install Roo Code
2. Locate the MCP configuration file for Roo Code
3. Copy the contents of your DiffuGen's diffugen.json file into Roo Code's MCP configuration
4. Use DiffuGen through the AI assistant feature

Setting up with Cline

1. Download and install Cline
2. Copy the contents of your DiffuGen's diffugen.json file into Cline's MCP settings
3. Use DiffuGen through the AI chat or command interf

…

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: CLOUDWERX-DEV
• Source: CLOUDWERX-DEV/DiffuGen
• License: MIT
• Homepage: http://cloudwerx.dev

Install and usage instructions live in the source repository linked above.