# Kelly Agent Observability

> Agent Fleet Observability Desk App-in-Skill — local mock dashboard visualizing a fleet of LLM agents running behind a shared AI gateway for your organization. Use when the user invokes $kelly-agent-observability or /kelly-agent-observability, wants to review agent health, call volume, latency, error rates, cost, or trace-level chain breaks across an agent fleet. This is a demo/reference dashboard…

- **Type:** Skill
- **Install:** `agentstack add skill-mr-kelly-skills-kelly-agent-observability`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [mr-kelly](https://agentstack.voostack.com/s/mr-kelly)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** MIT
- **Upstream author:** [mr-kelly](https://github.com/mr-kelly)
- **Source:** https://github.com/mr-kelly/skills/tree/main/skills/kelly-agent-observability
- **Website:** https://mr-kelly.github.io/skills/

## Install

```sh
agentstack add skill-mr-kelly-skills-kelly-agent-observability
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# Agent Fleet Observability Desk

## Overview

Use this skill as a local operator surface over a MOCK fleet of LLM agents
running behind a shared AI gateway for a generic organization. It aggregates
per-agent call volume, latency (p50/p95), error/failure rate, and cost into one
file-backed App-in-Skill dashboard with a Fleet Overview, a sortable Agent
Health table, and a Trace Detail view that highlights exactly where a chain of
tool calls broke.

Default interaction mode: App UI. Unless the user explicitly asks for chat-only
handling, seed mock telemetry if missing, start/reuse the local app with
`app/start.sh`, and give the actual local URL. Use chat-only mode only when the
user says "纯聊天", "chat only", "no UI", or similar.

This is a demo/reference dashboard: all data is generated locally by
`scripts/generate_fleet_data.ts`. There is no real gateway, no real agents, and
no external network calls anywhere in this skill.

## App UI Screenshots

  
    
    
  
  
    Fleet OverviewTotal calls, total cost, degraded/critical/healthy agent counts, and a per-agent sparkline card grid.
    Agent HealthSortable table with call volume, p50/p95 latency, error rate, cost, and a status badge per agent.
  
  
    
    
  
  
    Trace DetailOrdered step timeline for one trace; the step where the chain broke is visually flagged.
    Chinese UIFull zh-CN chrome via the language toggle or lang=zh.
  

## Boundary

- Local, read-only mock dashboard. The app reads and writes local files only.
- The only mutating endpoint is `POST /api/handoffs`, which appends a
  human-in-the-loop acknowledgement or investigation note to a local
  `app/.data/handoffs.jsonl` file. It never calls any remote system, agent
  gateway, or third-party API.
- Never reference any real company, brand, or production system. Agent names
  are generic archetypes (Booking Assistant, Support Triage, etc.) for a
  generic organization ("your organization", "the fleet").

## Local App

Start the dashboard with:

```bash
skills/kelly-agent-observability/app/start.sh
```

First run installs `hono` and `@hono/node-server`, then seeds mock telemetry
into `app/.data/fleet.json` if it doesn't already exist. The frontend is
zero-build vanilla JS/HTML/CSS. The app uses local HTTP on `127.0.0.1`,
preferring port `3100` through `4100`, or `KELLY_AGENT_OBS_UI_PORT` when set.

To regenerate mock telemetry at any time:

```bash
node skills/kelly-agent-observability/scripts/generate_fleet_data.ts
```

## Demo Mode

- `?demo=1` opens a deterministic, fully offline mock fleet (8 agents, 48h of
  hourly buckets, ~16 traces per agent) for documentation and screenshots,
  without touching `app/.data/fleet.json`.
- `?demo=agents` and `?demo=trace` select named mock scenes (agent health table,
  a broken trace).
- `lang=en` or `lang=zh` forces UI chrome language for screenshots.
- Demo API responses never read or write local files.

UI language: support English and Chinese chrome with `Auto` default.

## Data Model

Read `references/fleet-schema.md` before editing the app, scripts, or the
generator. Primary local files:

- `app/.data/fleet.json`: generated mock telemetry (agents, per-agent metrics,
  hourly buckets, traces). Regenerated by `scripts/generate_fleet_data.ts`.
- `app/.data/handoffs.jsonl`: append-only human-in-the-loop handoff log written
  by `POST /api/handoffs`.

## Views

- `#/overview`: total calls (24h), total cost (today), degraded/critical/
  healthy agent counts, and a per-agent card grid with a call-volume sparkline.
- `#/agents`: sortable Agent Health table (call volume, p50/p95 latency, error
  rate, cost, status badge).
- `#/agents/`: agent detail — metrics, sparkline, recent traces, and
  a handoff form (acknowledge / needs investigation).
- `#/traces/`: trace detail — ordered step timeline with the
  chain-break step visually flagged, and a handoff form.
- `#/handoffs`: history of recorded handoffs.
- `#/settings`: data provider and status-threshold summary.

## Safety

- Never invent a real incident or real company data; this is illustrative mock
  telemetry only.
- Keep all writes local and file-based (`app/.data/handoffs.jsonl`). No
  external network calls anywhere in this skill.
- Redact nothing sensitive is collected here — there are no credentials in
  this skill.
## Execution reports

Re-read the active provider's decisions immediately before any approved execution. Record each concrete operation, target, status, timestamp, and error in the provider-backed execution report; keep app actions local-only.

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [mr-kelly](https://github.com/mr-kelly)
- **Source:** [mr-kelly/skills](https://github.com/mr-kelly/skills)
- **License:** MIT
- **Homepage:** https://mr-kelly.github.io/skills/

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** no
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/skill-mr-kelly-skills-kelly-agent-observability
- Seller: https://agentstack.voostack.com/s/mr-kelly
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.