# Local Ai App Integration > >- - **Type:** Skill - **Install:** `agentstack add skill-amd-skills-local-ai-app-integration` - **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior - **Seller:** [amd](https://agentstack.voostack.com/s/amd) - **Installs:** 0 - **Category:** [Agent Skills](https://agentstack.voostack.com/c/agent-skills) - **Latest version:** 0.1.0 - **License:** MIT - **Upstream author:** [amd](https://github.com/amd) - **Source:** https://github.com/amd/skills/tree/main/skills/local-ai-app-integration - **Website:** https://developer.amd.com/ ## Install ```sh agentstack add skill-amd-skills-local-ai-app-integration ``` Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent. ## About # Local AI App Integration (Embeddable Lemonade) Add a local AI mode to an existing app that already talks to a cloud AI API (OpenAI, Anthropic, or Ollama-compatible). The app launches `lemond`, the Embeddable Lemonade binary, as a private subprocess and the existing client talks to it on `http://localhost:PORT/api/v1`. The user gets local, private, hardware-optimized inference (CPU, AMD iGPU/dGPU, XDNA2 NPU) with no separate install. **What you'll end up with:** one new launcher module (~30 lines), one config change to the existing HTTP client (base URL + API key), one vendored binary under `vendor/lemonade/`. ## When this skill is the right tool Use this skill when **all** of the following are true: - The app already calls a cloud AI service over HTTP (OpenAI Chat Completions, Anthropic Messages, or Ollama). - The user wants that AI to run on the end-user's PC, with the AI engine bundled into the app, not as a separate user install. - The target platform is Windows x64 or Linux x64 (macOS embeddable is in beta). If the user instead wants a **system-wide** Lemonade Server (one install, shared across apps), do not use this skill; point them at `https://lemonade-server.ai/install_options.html` and the standard OpenAI base URL `http://localhost:13305/api/v1`. ## The opinionated path This skill follows one fixed sequence. Do not deviate without a stated reason. ``` [ ] 1. Survey the app's current AI integration [ ] 2. Pick a model + backend profile [ ] 3. Place Embeddable Lemonade in the app's tree (full package, not just the binary) [ ] 4. Add a `lemond` launcher (subprocess + API key + port + per-stage logging) [ ] 5. Re-point the existing client at lemond (set HTTP timeout to 120s) [ ] 6. Wait for /api/v1/health, install backend, then PULL the model before first use [ ] 7. Wire shutdown and error recovery ``` Track progress against this checklist. Move on only when each step verifies. > **Log every stage.** A local integration has many silent failure points — > spawn, health, backend install, model download, first inference. Without a > log line at each transition, "nothing happened" is indistinguishable from > "broke at stage 3." Emit one clear line per stage as you build (see > [Step 4](#step-4-add-a-lemond-launcher)); the most common dead-end in this > integration — a blank result with no error — is invisible without them. --- ## Step 1: Survey the app Find every place the app currently calls a cloud AI API. Search the repo for: - `openai`, `OpenAI(`, `chat.completions`, `responses.create` - `anthropic`, `Anthropic(`, `messages.create` - `api.openai.com`, `api.anthropic.com`, `localhost:11434` (Ollama) - `OPENAI_API_KEY`, `ANTHROPIC_API_KEY` Record three things before continuing: 1. **Client library and language** (e.g., `openai-python`, `openai-node`, `@anthropic-ai/sdk`, `go-openai`, raw `fetch`). 2. **Modalities used:** text chat, tool calling, embeddings, image gen, transcription, TTS. This drives the model + backend choice in Step 2. 3. **One single place** where the base URL and API key are constructed. If there isn't one, refactor to one before going further. Local-mode toggling must flip exactly one config object. 4. **Any API-key gating** that blocks the app before a key is entered (onboarding walls, validators that reject empty keys, startup checks that disable AI until a key exists). Note each one — Step 5 bypasses them in local mode. ## Step 2: Pick a model + backend profile Choose **one** default profile based on the app's primary modality. Do not ship a buffet. Ship one good default and document how the user can override it. | App's primary need | Default model | Recipe | Why | |---|---|---|---| | General chat / assistant | `Qwen3-4B-GGUF` | `llamacpp` | Small, fast, good tool calling, fits 8GB systems | | Coding assistant | `Qwen2.5-Coder-7B-Instruct-GGUF` | `llamacpp` | Strong code, runs on iGPU | | Vision / multimodal chat | `Gemma-4-E2B-it-GGUF` | `llamacpp` | Small multimodal default | | NPU-first on Ryzen AI | `Llama-3.2-3B-Instruct-Hybrid` | `ryzenai-llm` | XDNA2 NPU on Windows | | Speech-to-text (Windows) | `Whisper-Large-v3-Turbo` | `whispercpp` | One model; probe picks NPU → iGPU/dGPU → CPU automatically | | Speech-to-text (Linux NPU) | `whisper-v3-turbo-FLM` | `flm` | Linux NPU path; falls back to `whispercpp` iGPU/CPU off-NPU | | Text-to-speech | `kokoro-v1` | `kokoro` | CPU-only, low latency | | Image generation | `SDXL-Turbo` | `sd-cpp` | Single-step generation | For the LLM backend, default to `llamacpp` and let `lemond` pick `rocm` → `vulkan` → `cpu` automatically by leaving `llamacpp_backend` unset. Override only if the app has hard hardware requirements. **Scope: this skill selects a backend once at integration time on the developer's machine.** Runtime fallback based on the end user's hardware is out of scope. Bundle `vulkan` as the universal fallback so the app works on any machine. If the dev machine has an NPU and the chosen recipe supports it, the skill will use the NPU backend — otherwise it falls back to `vulkan`. > **Note:** having an NPU does not mean every recipe supports NPU. Confirm > the recipe/backend pair is `installed` or `installable` via > `GET /api/v1/system-info` before committing to it. See > [reference.md](reference.md#hardware-probing-with-v1system-info) for > per-recipe decision rules. For more options and tradeoffs, see [reference.md](reference.md). ## Step 3: Place Embeddable Lemonade in the app's tree and install backends **Get the embeddable artifact** from the latest Lemonade release: ``` https://github.com/lemonade-sdk/lemonade/releases/latest ``` Download the file matching your target OS: - Windows: `lemonade-embeddable-{VERSION}-windows-x64.zip` - Linux: `lemonade-embeddable-{VERSION}-ubuntu-x64.tar.gz` > **Don't hand-build the download URL from the tag.** The git tag carries a > leading `v` (e.g. `v10.8.0`) but the asset filename strips it > (`lemonade-embeddable-10.8.0-...`), so using the tag verbatim 404s. Ask the > GitHub API for the asset by its stable name pattern and use the URL it > returns, as below — this stays correct across version and naming changes. **First, create the target directory** — it does not exist in a fresh repo: ```powershell # Windows New-Item -ItemType Directory -Force vendor\lemonade ``` ```bash # Linux mkdir -p vendor/lemonade ``` Then download and unpack on Windows (PowerShell): ```powershell $rel = Invoke-RestMethod https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest $asset = $rel.assets | Where-Object { $_.name -like "lemonade-embeddable-*-windows-x64.zip" } | Select-Object -First 1 Invoke-WebRequest $asset.browser_download_url -OutFile lemond.zip Expand-Archive lemond.zip -DestinationPath "$env:TEMP\lemond-unpack" $folder = $asset.name -replace '\.zip$','' # unpacked dir = asset name without .zip Copy-Item -Recurse "$env:TEMP\lemond-unpack\$folder\*" vendor\lemonade\ # Sanity check: resources/ must be nested under vendor\lemonade\ (not flattened) if (-not (Test-Path vendor\lemonade\resources\*.json)) { throw "resources/ missing — re-extract and copy again" } ``` On Linux (bash): ```bash URL=$(curl -s https://api.github.com/repos/lemonade-sdk/lemonade/releases/latest \ | grep browser_download_url | grep ubuntu-x64.tar.gz | cut -d'"' -f4) curl -L "$URL" | tar -xz --strip-components=1 -C vendor/lemonade ``` > **Copy the full package, not just the binary.** The archive contains > `lemond[.exe]`, `lemonade[.exe]`, `LICENSE`, and `resources/`. The > `resources/` directory is required — without it lemond starts and passes the > health check but fails on every model and backend request. Copying only the > binary produces a server that looks healthy but cannot function. > **`lemond` vs `lemonade` CLI:** `lemond` is the embedded server binary that > ships with the app. The `lemonade` CLI is a separate packaging tool used > only during development/build time to install backends. Install it once on > the developer machine with `pip install lemonade-sdk`. The expected layout **after setup** (first run + backend install). A freshly unzipped package contains only `lemond[.exe]`, `lemonade[.exe]`, `LICENSE`, and `resources/` — the items below are created later, as their comments note: ``` vendor/lemonade/ lemond[.exe] # the only binary the app ships LICENSE config.json # generated on first run; commit a seed copy resources/ server_models.json # do not edit; use GET /api/v1/models at runtime backend_versions.json bin/ # backends bundled at packaging time llamacpp/vulkan/llama-server[.exe] models/ # pre-bundled model weights (optional) models--unsloth--Qwen3-4B-GGUF/ ``` > **`server_models.json`:** Do not edit or rely on this file. It can be stale. > The only authoritative model list is `GET /api/v1/models` on a running > `lemond` instance with the backend already installed. **Bundle decisions: pick deliberately** - **Backends:** Bundle `llamacpp:vulkan` at packaging time (works on every GPU). Install `llamacpp:rocm` at first run on supported AMD systems via `POST /api/v1/install` after probing `GET /api/v1/system-info`. Never ship every backend, or the artifact balloons. - **Models:** Either bundle the default model under `models/` (offline install, larger installer) **or** pull on first run with `POST /api/v1/pull` (smaller installer, needs network). Pick one and document it. - **`models_dir`:** Set to `./models` in `config.json` to keep weights private to the app. Leave as `auto` only if the user explicitly wants to share weights with other apps. **Backend install timing — two distinct paths:** > **Packaging time** (developer machine, before bundling): > ``` > lemonade backends install llamacpp:vulkan > lemonade backends install flm:npu # Windows NPU path only > ``` > This bakes the backend binaries into `vendor/lemonade/bin/` before the app > ships. `lemond` does not need to be running. > > **First-run / runtime** (user's machine, after `lemond` is running): > ```http > POST /api/v1/install > {"recipe": "llamacpp", "backend": "rocm"} > ``` > Use this for hardware-specific backends (e.g. `llamacpp:rocm`) that cannot > be bundled universally. `lemond` must already be running (Step 4 complete). ## Step 4: Add a `lemond` launcher The launcher is a thin process supervisor. Its only jobs: 1. Generate a fresh random API key per app launch. 2. Pick a free localhost port. 3. Spawn `lemond --port ` with `LEMONADE_API_KEY` set. 4. Expose the chosen `port` and `key` to the rest of the app. > **Log one line per lifecycle stage.** Build the logging in from the start — > not as an afterthought when something breaks. Each silent transition needs a > visible marker so a failure points at the exact stage. Aim for: > > ``` > [lemond] Starting on port > [lemond] Healthy on port > [lemond] : installed (or: already installed / install failed) > [lemond] Pulling model ... then: Model ready (or: pull returned ) > [local] result: (first inference output — empty string here = unpulled model) > ``` > > Logging the **first inference result verbatim** is what turns the > silent-empty failure (Step 6) from a multi-hour mystery into a one-line > diagnosis. Route these through the app's normal logging so they can be quieted > for release. > **Dev-mode file watchers:** If the app runs with a file watcher (Tauri, > Electron, Next.js, Vite, etc.) that watches the source tree, ensure > `vendor/lemonade/` is excluded from the watched paths. Lemond writes config > and cache files at runtime; a watcher that picks these up will restart the > app, kill the lemond subprocess, and spawn a new one on a new port — > silently breaking any in-flight transcription. Add `vendor/` (or the > equivalent) to the watcher's ignore list before testing. The launcher logic in pseudocode (full Python and Node.js implementations in [reference.md](reference.md#reference-launchers)): ``` port = bind("127.0.0.1:0"), read port, close socket key = random_bytes(32) proc = spawn(lemond_bin, [lemond_dir, "--port", port], env={LEMONADE_API_KEY: key}) poll GET /api/v1/health with Bearer key, retry for 90s, 250ms interval return proc, key, port # On failure: kill proc, pick new port, retry up to 3 times # On app exit: proc.kill() (Windows) / proc.terminate() (Unix), then wait() ``` ## Step 5: Re-point the existing client at `lemond` Change exactly two values in the app's existing client config: the base URL and the API key. Nothing else. | Existing client | New `base_url` | New auth | |---|---|---| | `openai-python` / `openai-node` | `http://127.0.0.1:{port}/api/v1` | `api_key=key` | | `@anthropic-ai/sdk` | `http://127.0.0.1:{port}/api/v1` | `apiKey: key` (Lemonade serves the Anthropic API too) | | Raw `fetch` / `requests` | same as above | `Authorization: Bearer {key}` header | | Ollama-compatible code | `http://127.0.0.1:{port}/api/v0` | none required, but pass the key anyway | The model identifier on requests stays a Lemonade model name (e.g. `Qwen3-4B-GGUF`), not the cloud name. **Local mode needs no cloud API key — at all.** This is a defining property of local mode, not an edge case: there is no cloud service to authenticate to, so nothing should ever ask the user for a key. Any onboarding wall, validator, or startup check that demands one must not block local-mode users. Concretely: - Skip or auto-satisfy the key-entry screen in local mode. - Treat local mode as already-authorized in every validation path — an empty-key check must short-circuit to "valid" when the active mode is local, never throw "API key not configured". - Re-enable the gate **only** for cloud mode. The `lemond` key from Step 4 is generated internally by the launcher and used only for the local loopback connection, so the user never sees or enters one; any UI placeholder (e.g. `"local"`) is fine. Flipping into local mode should never strand the user on a key-entry wall. **Set the HTTP client timeout to at least 120 seconds.** The default timeout on most HTTP clients (30s) is shorter than the time lemond takes to load a model on first use. A silent timeout looks identical to a broken integration — the request fires, nothing comes back, and the UI shows nothing. 120s covers first-run model load on any supported hardware. **Python (openai) example:** ```python from openai import OpenAI import httpx proc, key, port = start_lemond() client = OpenAI( base_url=f"http://127.0.0.1:{port}/api/v1", api_key=key, http_client=httpx.Client(timeout=120.0), # covers first-run model load ) resp = client.chat.completions.create( model="Qwen3-4B-GGUF", messages=[{"role": "user", "content": "Hello"}], ) ``` ## Step 6: Health, backend, then pull the model — *before* first inference `GET /api/v1/health` returning 200 means the **server** is up. It does **not** mean inference will work. Before the first real request succeeds, three more things must be true: the backend for your modality is installed, the model's weights are **downloaded to disk**, and (on the first call) the model is loaded into memory. Treating health=200 as "ready" is the single biggest cause of a broken-looking integration. **Do not call `POST /api/v1/load` at startup.** Lemond lazy-loads the model into memory on the first inference request and handles that step on its own. Pre-loading is unreliable across lemond versions (the `/load` request body shape has changed between releases) and a malformed call can crash or destabilise the server before the user takes any action. Loading is the one step you let lemond do lazily — pulling is not. ### Pull the model so it exists on disk Lazy-load only loads weights that are **already downloaded**. If the model was never pulled, the first inference does not error — lemond returns an emp … ## Source & license This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code. - **Author:** [amd](https://github.com/amd) - **Source:** [amd/skills](https://github.com/amd/skills) - **License:** MIT - **Homepage:** https://developer.amd.com/ Install and usage instructions live in the source repository linked above. ## Pricing - **Free** — Free ## Security capabilities Automated source analysis of v0.1.0 — what this tool can access: - **Network access:** yes - **Filesystem access:** no - **Shell / process execution:** yes - **Environment & secrets:** no - **Dynamic code execution:** no *"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.* ## Versions - **0.1.0** — security scan: passed — Imported from the upstream source. ## Links - Listing page: https://agentstack.voostack.com/l/skill-amd-skills-local-ai-app-integration - Seller: https://agentstack.voostack.com/s/amd - Browse the marketplace: https://agentstack.voostack.com/browse --- Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.