Esp32 Cam Ai

ESP32-CAM MCP Server

Transform your ESP32-CAM into a powerful, remotely controllable AI-enabled camera system!

[](https://github.com/rzeldent/esp32-cam-ai/actions/workflows/main.yml)

[](assets/images/esp32-cam-ai.png)

Overview

This project transforms an ESP32-CAM into a remotely controllable MCP server that can capture images, control LEDs, manage flash lighting, and provide system diagnostics. The server exposes these capabilities through the Model Context Protocol, making it easy to integrate with AI assistants and automation systems.

Brief: Use Copilot or other AI digital assistants, like AI-Toolkit (in VSCode), Home Assistant (HA) or Node-Red, to use your ESP32-CAM, getting information (camera, wifi- or system state) or set GPIO's (led, flash)

Features

Hardware Control Tools

• LED Control: Turn the ESP32-CAM's built-in LED on/off
• Flash Control: Trigger camera flash with configurable duration (5-100ms)
• Camera Capture: Take photos with optional flash support (optimized for 75°C | Critical: >85°C

Memory Health Indicators:

• Healthy: >100KB free | Moderate: 50-100KB | High pressure: <50KB | Critical: <30KB

Usage Examples

Direct HTTP Requests

# Capture image with flash
$body = '{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": {"name": "capture", "arguments": {"flash": "on"}}}'
Invoke-RestMethod -Uri "http://192.168.1.132/" -Method Post -Body $body -ContentType "application/json"

Integration with AI Assistants

The MCP server can be integrated with AI assistants that support the Model Context Protocol:

1. Configure MCP Client (mcp.json):

{
  "servers": {
    "esp32-cam-ai": {
      "type": "http",
      "url": "http://192.168.1.132"
    }
  }
}

1. Use Natural Language Commands:

• "Take a picture with the ESP32-CAM"
• "Turn on the LED"
• "Check the WiFi status"
• "Flash the camera"

Automation Integration

• AI-Toolkit in VSCode: Create prompts and use custom MCP's. https://marketplace.visualstudio.com/items?itemName=ms-windows-ai-studio.windows-ai-studio
• Home Assistant: Create automations triggered by camera captures
• Node-RED: Build visual workflows with camera and LED control
• Custom Applications: Integrate via standard HTTP requests

Architecture

Core Components

┌───────────────────┐    ┌───────────────────┐    ┌───────────────────┐
│    MCP Protocol   │────│   Tool Handlers   │────│   Hardware Layer  │
│                   │    │                   │    │                   │
│ • Initialization  │    │ • LED Control     │    │ • Camera (OV2640) │
│ • Tools List      │    │ • Flash Control   │    │ • GPIO Control    │
│ • Tools Call      │    │ • Capture         │    │ • WiFi Module     │
│ • Notifications   │    │ • Status Check    │    │ • Flash Storage   │
└───────────────────┘    └───────────────────┘    └───────────────────┘
  │                          │                      │
  └──────────────────────────┼──────────────────────┘
              │
          ┌───────────────────┐
          │    Web Server     │
          │                   │
          │ • HTTP Endpoint   │
          │ • JSON Parsing    │
          │ • Error Handling  │
          │ • Response Gen.   │
          └───────────────────┘

Code Structure

esp32-cam-ai/
├── src/
│   └── main.cpp              # Main application code
├── include/
│   └── camera_config.h       # Camera configurations
├── lib/
│   └── mcp/                  # MCP protocol implementation
│       ├── mcp.h
│       └── mcp.cpp
├── .vscode/
│   └── mcp.json             # MCP client configuration
└── platformio.ini           # Build configuration

Reliability Features

WiFi Management

• Auto-reconnection: Automatic retry with exponential backoff
• Connection monitoring: Periodic status checks every 5 seconds
• Recovery mechanisms: System restart after max failed attempts
• Event handling: Proper WiFi event management

System Stability

• Watchdog Timer: 10-second timeout prevents system hangs
• Memory Management: Proper cleanup of camera frame buffers
• Error Handling: Comprehensive error codes and messages
• OTA Support: Remote firmware updates for maintenance

Camera Management

• Initialization Checks: Verify camera before operations
• Frame Buffer Management: Proper allocation and cleanup
• Flash Timing: Synchronized flash and capture timing
• Quality Settings: Configurable resolution and compression

Troubleshooting

Common Issues

Camera Not Working

// Check camera initialization in serial output
Camera init failed with error 0x[code]

Solutions:

• Verify camera connections
• Check power supply (adequate current)
• Try different camera configurations

WiFi Connection Problems

// Check WiFi credentials and network
Failed to connect to WiFi. Error code: [code]

Solutions:

• Verify SSID and password in .env file
• Ensure .env file exists in project root directory
• Check 2.4GHz network availability (ESP32 doesn't support 5GHz)
• Ensure adequate signal strength
• Verify .env file format is correct (no spaces around =)

Build Errors

WIFI_SSID is not defined. Please define it in your environment variables or in the code.
WIFI_PASSWORD is not defined. Please define it in your environment variables or in the code.

Solutions:

• Create a .env file in the project root directory
• Add WiFi credentials to .env file in the correct format:

  WIFI_SSID=YourNetworkName
  WIFI_PASSWORD=YourPassword
  ```

- Ensure no spaces around the `=` sign
- Use quotes around values containing special characters or spaces

#### Memory Issues

```cpp
// Monitor heap usage
Free Heap: [bytes] bytes

Solutions:

• Reduce image quality/resolution to stay under 4KB limit
• Increase frame buffer count
• Check for memory leaks

Serial Debug Commands

Monitor the serial output for diagnostic information:

CPU Freq: 240 MHz
Free heap: 189092 bytes
WiFi got IP address: 192.168.1.132
Camera initialized: Yes

Advanced Usage

Custom Camera Settings

Modify camera_config.h for specific requirements:

// High quality settings (may exceed 4KB limit)
.frame_size = FRAMESIZE_SVGA,  // 800x600
.jpeg_quality = 8,             // Higher quality

// Optimized for 4KB limit
.frame_size = FRAMESIZE_QVGA,  // 320x240
.jpeg_quality = 25,            // Lower quality, smaller files

Adding Custom Tools

Extend the MCP server with additional tools:

1. Add tool definition in handle_tools_list()
2. Implement tool handler function
3. Register in handle_tools_call()

Integration Examples

Python Client

import requests
import base64
import json

def capture_image(esp32_ip, use_flash=False):
    url = f"http://{esp32_ip}/"
    payload = {
        "jsonrpc": "2.0",
        "id": 1,
        "method": "tools/call",
        "params": {
            "name": "capture",
            "arguments": {"flash": "on" if use_flash else "off"}
        }
    }
    
    response = requests.post(url, json=payload)
    data = response.json()
    
    if "result" in data:
        # Extract base64 image (under 4KB)
        image_data = data["result"]["content"][1]["data"]
        image_bytes = base64.b64decode(image_data)
        
        with open("captured_image.jpg", "wb") as f:
            f.write(image_bytes)
        
        print("Image saved as captured_image.jpg")
    else:
        print(f"Error: {data.get('error', {}).get('message', 'Unknown error')}")

# Usage
capture_image("192.168.1.132", use_flash=True)

Node.js Integration

const axios = require('axios');
const fs = require('fs');

async function captureImage(esp32IP, useFlash = false) {
    const payload = {
        jsonrpc: "2.0",
        id: 1,
        method: "tools/call",
        params: {
            name: "capture",
            arguments: { flash: useFlash ? "on" : "off" }
        }
    };
    
    try {
        const response = await axios.post(`http://${esp32IP}/`, payload);
        const imageData = response.data.result.content[1].data;
        const imageBuffer = Buffer.from(imageData, 'base64');
        
        fs.writeFileSync('captured_image.jpg', imageBuffer);
        console.log('Image saved as captured_image.jpg');
    } catch (error) {
        console.error('Error capturing image:', error.message);
    }
}

// Usage
captureImage('192.168.1.132', true);

Contributing

1. Fork the repository
2. Create feature branch: git checkout -b feature/new-tool
3. Commit changes: git commit -am 'Add new MCP tool'
4. Push to branch: git push origin feature/new-tool
5. Create Pull Request

Development Guidelines

• Follow existing code style and patterns
• Add proper error handling for new features
• Update documentation for new tools
• Test thoroughly on actual hardware

License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

Acknowledgments

• ESP32 Community for excellent camera libraries
• Model Context Protocol specification authors
• PlatformIO for the excellent development environment
• ArduinoJson library for efficient JSON handling

Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: This README and inline code comments

Performance Metrics

Typical Performance

• Image Capture: ~2-3 seconds for QVGA (320x240) JPEG
• Tool Response Time: <500ms for LED/Flash control
• WiFi Reconnection: ~10-15 seconds recovery time
• Memory Usage: ~200KB free heap during normal operation
• Network Latency: <100ms for local network requests

Resource Usage

• Flash Storage: ~1.2MB for compiled firmware
• RAM Usage: ~100KB for base functionality + camera buffers
• CPU Usage: <5% during idle, ~30% during image capture
• Network Bandwidth: ~50KB per image (QVGA quality 12, under 4KB base64)

Optimization Tips

• Lower JPEG Quality: Reduces file size and capture time
• Smaller Frame Size: Improves performance and memory usage
• Reduce Frame Buffer Count: Saves memory but may affect stability
• WiFi Power Management: Balance between performance and power consumption

Security Considerations

Network Security

• Open HTTP Server: No built-in authentication (add custom authentication if needed)
• Local Network Access: Device accessible to all network clients
• Unencrypted Communication: Consider HTTPS for sensitive deployments
• IP Address Exposure: Device IP visible on network scans

Physical Security

• Camera Placement: Position device appropriately for intended monitoring
• Access Control: Secure physical access to device and programming pins
• Power Supply: Ensure stable power to prevent data corruption
• SD Card: If used, secure against unauthorized access

Recommended Security Measures

1. Network Segmentation: Use dedicated IoT VLAN
2. Firewall Rules: Restrict access to specific IP ranges
3. Regular Updates: Keep firmware updated via OTA
4. Access Logging: Monitor connection attempts
5. Custom Authentication: Add API key or token validation

Privacy Considerations

• Image Data: Captured images transmitted as base64 over HTTP (under 4KB)
• Local Processing: No cloud storage by default
• Data Retention: Images not stored on device (unless explicitly saved)
• Network Monitoring: Be aware of network traffic for image transfers

---

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: rzeldent
• Source: rzeldent/esp32-cam-ai
• License: MIT
• Homepage: https://github.com/rzeldent/ESP32-CAM-AI

Install and usage instructions live in the source repository linked above.