# Llm Backends

> Discover, audit and use local LLM servers (LM Studio, Ollama, LocalAI, vLLM, llama.cpp) on this machine or the network to offload work from the cloud and save tokens. Use when the user mentions local models, LM Studio, Ollama, "run it locally", token savings via local hardware, or wants to know what models their machine can run.

- **Type:** Skill
- **Install:** `agentstack add skill-zedarvates-botte-secrete-llm-backends`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [zedarvates](https://agentstack.voostack.com/s/zedarvates)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [zedarvates](https://github.com/zedarvates)
- **Source:** https://github.com/zedarvates/botte-secrete/tree/main/skills/llm_backends
- **Website:** https://github.com/zedarvates/botte-secrete

## Install

```sh
agentstack add skill-zedarvates-botte-secrete-llm-backends
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# llm_backends — Local LLM discovery, audit & routing

Turn idle local hardware into a token-saving tier. Every task served by a local
model is a task **not** billed to the cloud.

## When to use

- The user mentions **LM Studio, Ollama, LocalAI, vLLM, llama.cpp**, "local model",
  "run it locally", or "use my GPU".
- You need to know **what models are reachable** (this machine or the network).
- A task is cheap/local-suitable (classification, extraction, short summary,
  routing, spell-check, simple Q&A) and could skip the cloud entirely.
- The user has **no** local model yet and wants step-by-step, hardware-aware setup.

## Quick commands

```bash
# Discover + register backends (writes configs/llm-endpoints.json)
python -m skills.llm_backends.cli scan                 # localhost only
python -m skills.llm_backends.cli scan --subnet        # sweep local /24
python -m skills.llm_backends.cli scan 192.168.1.47    # specific host(s)

# What's registered?
python -m skills.llm_backends.cli list

# Audit: are local models used? what can this machine run? next steps?
python -m skills.llm_backends.cli audit --fresh

# Run a prompt locally (0 cloud tokens)
python -m skills.llm_backends.cli chat "classify: bug or feature?" --max-tokens 128

# Suggest a local model for a project (adaptive per project type)
python -m skills.llm_backends.cli profile ~/my-project
```

## Programmatic use

```python
from skills.llm_backends import registry, quick_chat, audit

registry.refresh()                       # discover + persist
best = registry.best_chat_backend()      # lowest-latency chat backend
model = registry.preferred_model(best)   # coder/instruct over voice/reasoning

res = quick_chat("summarize in 1 line: ...", max_tokens=200)
print(res.text, res.total_tokens)        # all local — no cloud cost
```

## Supported backends

| Backend | Default port | API |
|---------|-------------|-----|
| LM Studio | 1234 | OpenAI `/v1` |
| Ollama | 11434 | native `/api/tags` + OpenAI `/v1` |
| LocalAI | 8080 | OpenAI `/v1` |
| vLLM | 8000 | OpenAI `/v1` |
| Jan / KoboldCpp / text-gen-webui | 1337 / 5001 / 5000 | OpenAI `/v1` |
| ComfyUI | 8188 | image gen |
| Qdrant | 6333 | vector search |

## How it saves tokens

1. **Audit** finds reachable local backends and profiles RAM/VRAM/GPU.
2. **Route** sends local-suitable tasks (see `tiered_router` L0/L1) to a local model.
3. **Call** runs them via the OpenAI-compatible client — zero cloud tokens.
4. **Onboard** users with no local model, recommending the largest model their
   hardware can run and the right server to install.

Related: [[llm_mcp]] (MCP tools for agents), `tiered_router` (cost tiers),
`local_router` (task→backend mapping), `response_cache` (skip repeated calls).
```

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [zedarvates](https://github.com/zedarvates)
- **Source:** [zedarvates/botte-secrete](https://github.com/zedarvates/botte-secrete)
- **License:** MIT
- **Homepage:** https://github.com/zedarvates/botte-secrete

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-zedarvates-botte-secrete-llm-backends
- Seller: https://agentstack.voostack.com/s/zedarvates
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.