Gemini Proxy

gemini-cli-openai-proxy

A high-performance, stateless OpenAI-compatible API gateway proxy for Google's official gemini-cli.

This proxy translates incoming /v1/chat/completions (streaming/non-streaming) and /v1/models REST payloads into isolated, headless gemini-cli subprocess executions and streams Server-Sent Events (SSE) directly back to any standard client (e.g., Open WebUI, VS Code Continue, Emacs gptel).

---

Why Rust? (Architectural Comparison with Node.js)

When translating an architecture from a JavaScript/Node.js ecosystem into a systems-programming language like Rust, we made fundamental structural shifts under the hood to maximize performance, safety, and simplicity.

| Metric / Aspect | Traditional Node.js Proxy | Our Rust Proxy Design | | :--- | :--- | :--- | | Integration Model | Downloads gemini-cli source as a Git submodule and imports internal code directly. Runs server and Gemini logic in the same JS memory bubble. | Acts as an independent "process manager." Spawns your already-installed gemini-cli as an isolated background process, communicating via stdin/stdout pipes. | | Footprint & Deployment | Requires Node.js, npm install (hundreds of MB of node_modules), and TS compilation. Idles at 40MB - 100MB RAM. | Single, hyper-lean standalone binary. No node_modules required to run the server. Idles at 3MB - 5MB RAM with zero GC latency. | | Protocol Translation | Bypasses translation by importing CLI code directly, calling Google's SDK directly inside JS. | Speaks OpenAI REST on the front end (to Open WebUI/Emacs) and cleanly maps those payloads into headless gemini prompt executions using stream-json line-by-line streams on the back end. |

In short, while a Node.js proxy is a heavily integrated script, this Rust version is a lightweight, universal, secure system daemon that wraps the CLI from the outside!

---

Features

• Stateless & Parallel: Spawns a lightweight, headless subprocess per-request. Allows 100% concurrent execution with zero thread-locking or session contamination.
• Native Tool Support: Integrates seamlessly with gemini-cli's built-in tool execution engine. External clients (like Emacs gptel) don't need to define tools—they can natively ask the LLM to search the web, read files, or run commands.
• Secure Sandbox Execution: Forces strict isolation by generating a unique, disposable temporary directory (/tmp/gemini_proxy_xxxx) for every request, preventing the LLM from attempting lateral local filesystem read/write actions on your host machine.
• Configurable Tool Security: Replaces insecure yolo mode with auto_edit by default, protecting your system from destructive headless shell commands while still enabling file system actions strictly inside the temporary sandbox.
• Workspace Trust Bypass: Automatically overrides headless directory trust checks (--skip-trust).
• Secure Network Binding: Defaults strictly to 127.0.0.1 (localhost), with optional binding override via environment variables.
• Real-time Performance Profiling: Logs process spawn duration, Time-to-First-Token (TTFT), and average generation speed (tokens/second) directly to stdout for every query.

---

Requirements

1. gemini (Official Google Gemini CLI binary installed in your system PATH).
2. Active GCP Authentication:

``bash gcloud auth login gcloud auth application-default login ``

---

Supported Models

The proxy exposes the active Vertex AI endpoint matrix:

| Model ID | Target Endpoint | Description | | :--- | :--- | :--- | | gemini-cli | gemini-3-flash | Default fast, general-purpose model | | gemini-3-flash | gemini-3-flash | Standard high-speed generation model | | gemini-3.1-flash-lite | gemini-3.1-flash-lite | Ultra-fast lightweight model | | gemini-1.5-pro | gemini-1.5-pro | Stable legacy high-reasoning Pro model | | gemini-2.5-pro | gemini-2.5-pro | Advanced high-reasoning Pro model | | gemini-3.1-pro-preview | gemini-3.1-pro-preview | Live cutting-edge high-reasoning Pro model |

---

Building & Running

1. Run with secure defaults (localhost only)

cargo run --release

Server boots instantly on http://127.0.0.1:8765.

2. Run with custom interface and port

If you need to connect from containerized applications (like Dockerized Open WebUI) without exposing the port to your physical LAN, bind the proxy to your private virtual Docker bridge gateway IP:

BIND_ADDRESS=172.17.0.1 PORT=8080 cargo run --release

---

Tool Execution & Security

The proxy natively supports gemini-cli tools. However, because the proxy runs the CLI in a headless non-interactive environment (with standard input closed), any tool that requires a manual confirmation prompt will immediately receive an EOF signal and fail safely with "User denied execution."

To balance autonomy and security, you can configure the auto-approval behavior via the GEMINI_APPROVAL_MODE environment variable:

GEMINI_APPROVAL_MODE=auto_edit cargo run

Approval Modes

• auto_edit (Default & Recommended): Auto-approves safe file system operations (read/write/replace). gemini-cli confines these tools to the request's temporary sandbox—attempts to read or write absolute paths outside it (e.g. ~/.ssh/id_rsa) are rejected with Path not in workspace. Arbitrary shell commands will fail safely.
• yolo (Insecure): Auto-approves all tool calls, including shell commands. Use with extreme caution.
• plan: Read-only mode. No destructive tools are permitted.

Granular Tool Security (policy.yaml)

If you want to use --approval-mode auto_edit but still allow the LLM to run specific, safe shell commands headlessly (e.g., cargo check or npm test), you can utilize gemini-cli's Policy Engine.

Create a policy.yaml file in your global ~/.gemini/ directory or your project root:

version: 1
permissions:
  - tool: default_api:run_shell_command
    action: allow
    # Use regular expressions to explicitly allow safe commands without a prompt
    condition: ^(cargo check|npm test|ls -la)$

The proxy will automatically inherit this policy, allowing the LLM to execute these specific commands safely in the background while still blocking unknown or dangerous commands!

> Warning: Unlike the file edit tools, shell commands allowed here are not confined to the temporary sandbox—they run with full access to your home directory and host filesystem. Anchor every condition regex with ^...$ and keep the allowlist to commands that take no untrusted path arguments. A loose pattern like ^cat .*$ would let the model read ~/.ssh/id_rsa. For stronger isolation, run the underlying CLI with gemini --sandbox (Docker/Podman/Seatbelt).

---

Client Integration

Open WebUI (Docker)

Start your Open WebUI container with OPENAI_API_BASE_URL pointing to your host gateway:

docker run -d -p 3000:8080 \
  -e OPENAI_API_BASE_URL="http://host.docker.internal:8765/v1" \
  -e OPENAI_API_KEY="local" \
  --name open-webui \
  ghcr.io/open-webui/open-webui:main

1. Open Open WebUI (http://localhost:3000).
2. Go to Admin Settings -> Connections -> OpenAI API.
3. Click the circular refresh arrow icon next to the URL/Key fields.
4. Open WebUI will fetch the supported model list. Select your target Gemini model from the dropdown menu and start chatting.

Emacs gptel

If you use Emacs, you can configure gptel to use this local proxy as a custom OpenAI provider. Add the following Elisp configuration to your init file:

(use-package gptel
  :config
  (gptel-make-openai "GeminiProxy"
    :host "127.0.0.1:8765"
    :protocol "http"
    :key "sk-dummy"
    :stream t
    :models '(gemini-cli
              gemini-3-flash
              gemini-3.1-flash-lite
              gemini-1.5-pro
              gemini-2.5-pro
              gemini-3.1-pro-preview)))

---

Running as a macOS Background Daemon (launchd)

On macOS, the native and most robust way to run this proxy persistently in the background is through a user-level LaunchAgent using launchd. This ensures the proxy starts automatically whenever you log in.

1. Compile and Install the Binary

Build and install the binary directly using cargo install (this compiles in release mode and puts the binary inside ~/.cargo/bin/):

cargo install --path . --force

2. Create the LaunchAgent Plist Configuration

Create a configuration file at ~/Library/LaunchAgents/com.user.gemini-proxy.plist:

touch ~/Library/LaunchAgents/com.user.gemini-proxy.plist

Open this file and paste the following XML (replace YOUR_USERNAME with your actual macOS username, and verify your GOOGLE_CLOUD_PROJECT name):

    Label
    com.user.gemini-proxy
    ProgramArguments
    
        /Users/YOUR_USERNAME/.cargo/bin/gemini_proxy
    
    EnvironmentVariables
    
        PATH
        /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/homebrew/bin
        GOOGLE_CLOUD_PROJECT
        YOUR_GCP_PROJECT_ID
        BIND_ADDRESS
        127.0.0.1
        PORT
        8765
    
    RunAtLoad
    
    KeepAlive
    
    StandardOutPath
    /tmp/gemini-proxy.out.log
    StandardErrorPath
    /tmp/gemini-proxy.err.log

3. Load and Manage the Daemon

• Load and Start the daemon (enabling auto-start on login):

``bash launchctl load ~/Library/LaunchAgents/com.user.gemini-proxy.plist ``

• Stop the daemon from running:

``bash launchctl unload ~/Library/LaunchAgents/com.user.gemini-proxy.plist ``

• Check logs to verify performance and inspect requests:

``bash tail -f /tmp/gemini-proxy.out.log ``

4. Log Rotation & Disk Space Management (Optional)

To ensure these logs never fill up your filesystem, you can register them with macOS's native newsyslog utility for automatic, size-based log rotation and compression:

1. Create a newsyslog configuration file (requires sudo):

``bash sudo touch /etc/newsyslog.d/gemini-proxy.conf ``

1. Open the file and paste this rule (e.g., via sudo nano /etc/newsyslog.d/gemini-proxy.conf):

``text # logfilename mode count size when flags /tmp/gemini-proxy.*.log 644 3 5000 * J ``

This rule tells macOS to automatically rotate the logs the moment they exceed 5 MB (5000 KB), compress them with high-efficiency bzip2 compression (turning a 5MB text log into ~200KB), and keep only the last 3 historical backups before automatically purging older ones.

---

Acknowledgments & Credits

This project was inspired by the original Node.js implementation: Intelligent-Internet/gemini-cli-mcp-openai-bridge. Our Rust rewrite focuses on minimizing resource consumption, adding security-isolated sandboxing, providing native parallelization, and enabling high-resolution terminal performance profiling.

Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: FlerAlex
• Source: FlerAlex/gemini_proxy
• License: Apache-2.0
• Homepage: https://github.com/FlerAlex/gemini_proxy/releases

Install and usage instructions live in the source repository linked above.