Flowcmd Llm Recipes

LLM recipes for flowcmd steps

flowcmd is backend-agnostic: a step calls a model by running whatever CLI or HTTP client writes the response to stdout. These recipes are copy-pasteable run: blocks plus the env vars each backend expects. Drop them into a step, feed in {{ steps.previous.output }} as the prompt, and capture the model's text for downstream steps.

General rules:

• Use run: | (literal block scalar) whenever the command spans multiple lines.
• To embed a previous step's output into JSON safely, pipe it through jq -Rs . or use jq -n --arg. Never string-concatenate untrusted stdout directly into JSON.
• Keys and tokens come from the user's shell environment. flowcmd steps inherit that env untouched — no env: key is needed (or supported).
• Add retry: to any network-facing call. attempts: 3, delay: 2s-5s is a sensible default.

---

Ollama (local)

Prereqs: ollama on PATH, a model pulled (ollama pull llama3). Env vars: none required.

- name: ask-ollama
  run: |
    ollama run llama3 "Summarise this diff in 3 bullets:
    {{ steps.diff.output }}"

Ollama's CLI prints the completion to stdout directly — no JSON extraction needed.

---

Claude CLI

Prereqs: claude CLI authenticated. Env vars: whatever claude needs (its own config handles auth).

- name: ask-claude
  run: |
    claude -p "Generate a commit message for:
    {{ steps.diff.output }}"
  retry:
    attempts: 2
    delay: 5s

---

llama.cpp server

Prereqs: llama-server running (./llama-server -m model.gguf -c 4096). Default port: 8080. Endpoint: /completion returns {content: "..."} on success.

- name: ask-llamacpp
  run: |
    jq -n --arg p "Question: {{ steps.input.output }}" \
      '{prompt: $p, n_predict: 256, temperature: 0.2}' \
      | curl -s http://localhost:8080/completion \
          -H "Content-Type: application/json" \
          --data-binary @- \
      | jq -r '.content'
  retry:
    attempts: 3
    delay: 2s

llama.cpp also exposes an OpenAI-compatible /v1/chat/completions — see the "OpenAI-compatible" recipe below if you prefer that shape.

---

Hugging Face inference API

Prereqs: an HF account with an access token. Env vars: HF_TOKEN. Endpoint: https://api-inference.huggingface.co/models/.

- name: ask-hf
  run: |
    jq -n --arg t "{{ steps.input.output }}" '{inputs: $t}' \
      | curl -s https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.3 \
          -H "Authorization: Bearer $HF_TOKEN" \
          -H "Content-Type: application/json" \
          --data-binary @- \
      | jq -r '.[0].generated_text'
  retry:
    attempts: 3
    delay: 5s

Cold-start on HF is real — the first request after idle often returns 503 with {"error":"Model is loading"}. The retry block handles that.

---

Unsloth / vLLM / any OpenAI-compatible server

Unsloth, vLLM, llama.cpp (in --api mode), TGI, Ollama's /v1 endpoint, LM Studio, LocalAI — all expose the same OpenAI-compatible /v1/chat/completions shape.

Prereqs: server running locally or reachable over the network. Env vars (by convention): LLM_BASE_URL (default http://localhost:8000), LLM_API_KEY (many servers ignore the value but require the header).

- name: ask-oai-compatible
  run: |
    jq -n --arg q "{{ steps.input.output }}" '{
      model: "unsloth/Meta-Llama-3.1-8B-Instruct",
      messages: [{role: "user", content: $q}],
      temperature: 0.2
    }' \
      | curl -s "${LLM_BASE_URL:-http://localhost:8000}/v1/chat/completions" \
          -H "Authorization: Bearer ${LLM_API_KEY:-sk-local}" \
          -H "Content-Type: application/json" \
          --data-binary @- \
      | jq -r '.choices[0].message.content'
  retry:
    attempts: 2
    delay: 3s

Change the model field to whatever identifier your server advertises (often inspect with curl $LLM_BASE_URL/v1/models).

---

OpenAI

Prereqs: an API key. Env vars: OPENAI_API_KEY.

- name: ask-openai
  run: |
    jq -n --arg q "{{ steps.input.output }}" '{
      model: "gpt-4o-mini",
      messages: [{role: "user", content: $q}]
    }' \
      | curl -s https://api.openai.com/v1/chat/completions \
          -H "Authorization: Bearer $OPENAI_API_KEY" \
          -H "Content-Type: application/json" \
          --data-binary @- \
      | jq -r '.choices[0].message.content'
  retry:
    attempts: 3
    delay: 4s

---

Anthropic (direct API, without the Claude CLI)

Env vars: ANTHROPIC_API_KEY.

- name: ask-anthropic
  run: |
    jq -n --arg q "{{ steps.input.output }}" '{
      model: "claude-sonnet-4-5",
      max_tokens: 1024,
      messages: [{role: "user", content: $q}]
    }' \
      | curl -s https://api.anthropic.com/v1/messages \
          -H "x-api-key: $ANTHROPIC_API_KEY" \
          -H "anthropic-version: 2023-06-01" \
          -H "Content-Type: application/json" \
          --data-binary @- \
      | jq -r '.content[0].text'
  retry:
    attempts: 3
    delay: 4s

---

Generic HTTP endpoint

When the user has a bespoke endpoint, collect the same four things and plug them in:

1. URL
2. Auth header (if any)
3. Request body shape (what key carries the prompt?)
4. Response extraction (jq path to the generated text)

- name: ask-custom
  run: |
    jq -n --arg p "{{ steps.input.output }}" '{your_prompt_field: $p}' \
      | curl -s https://your-endpoint.example.com/path \
          -H "Authorization: Bearer $YOUR_TOKEN" \
          -H "Content-Type: application/json" \
          --data-binary @- \
      | jq -r '.your_response_field'
  retry:
    attempts: 3
    delay: 3s

---

Pitfalls that bite every time

1. Quotes in prompts. Step output often contains ", \, or newlines. Always build the JSON with jq -n --arg, never echo "{\"prompt\": \"…\"}".
2. Multi-line outputs. Use jq -Rs . to turn a whole file of text into a single JSON string.
3. 403 / 401. The token isn't set in the shell that launched flowcmd. Have the user export it and retry — flowcmd cannot read a .env file on its own.
4. Rate limits. Prefer retry: { attempts: 3+, delay: 5s+ } for hosted APIs. Backoff isn't exponential, but three tries with a 5s delay clears most transient 429s.
5. Empty response. Gate the next step with when: "{{ steps.ask-x.output != '' }}" so downstream work doesn't run on an empty string.

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: flowcmd
• Source: flowcmd/skill
• License: MIT

Install and usage instructions live in the source repository linked above.