# Java Obs Agent

> Framework-agnostic Java observability sidecar agent: chat with a live JVM via MCP/web UI (metrics, slow calls, errors, logs, thread dumps, method traces). Self-hosted, attachable (Arthas-style). Apache-2.0.

- **Type:** MCP server
- **Install:** `agentstack add mcp-twz007-java-obs-agent`
- **Verified:** Yes — security-reviewed for prompt injection and unsafe behavior
- **Seller:** [twz007](https://agentstack.voostack.com/s/twz007)
- **Installs:** 0
- **Category:** [AI & ML](https://agentstack.voostack.com/c/ai-and-ml)
- **Latest version:** 0.1.0
- **License:** Apache-2.0
- **Upstream author:** [twz007](https://github.com/twz007)
- **Source:** https://github.com/twz007/java-obs-agent

## Install

```sh
agentstack add mcp-twz007-java-obs-agent
```

Requires the [AgentStack CLI](https://agentstack.voostack.com/docs/cli). Works with Claude Code, Cursor, and any MCP-compatible agent.

## About

# java-obs-agent

[](LICENSE) [](README.zh-CN.md)

A **framework-agnostic Java observability sidecar agent** — drop a single `-javaagent` jar onto any Java (Servlet) service, then **chat with it** (via a built-in web UI, an MCP client like Claude Code, or curl) to understand the service's live state: request volume, latency, slow calls, errors, logs, thread dumps, method-level traces.

It's the thing between **Arthas** (in-JVM diagnostics, no AI/aggregation) and **SkyWalking** (APM with distributed traces, no conversational agent) — plus the **Datadog Bits AI** pattern (chat-with-your-service), but self-hosted and attachable to a single JVM.

> Built and validated end-to-end on a real Spring Boot service (the `interview-guide` project): attach → chat → "which route is slowest?" → agent calls its tools → answers with real metrics, then `trace_request` to pin the slow method in the call tree.

---

## What it does

Attach the agent to a running Java service (no restart needed — dynamic attach like Arthas). The agent runs **inside the target JVM**:

- Captures per-route request metrics (count, p50/p95/p99, error rate), slow calls, errors — into in-JVM ring buffers (zero external storage).
- Captures recent logs (Logback appender) + thread snapshots (JMX).
- Exposes a **MCP server** (JSON-RPC over HTTP) — any MCP client (Claude Code, Cursor) can chat with the service.
- Exposes a **built-in chat web UI** (`/chat`) — an OpenAI-compatible LLM (BYO API key) orchestrates the tools to answer your questions in natural language.
- On-demand **method-level diagnostics**: `watch_method` (capture a method's invocations) and `trace_request` (full method call-tree, Arthas-`trace` style).
- Leaves a **Source SPI** seam so a future SkyWalking/Tempo backend plugs in without changing tools (Phase 3).

## Quick start

### 1. Build

```bash
cd 
JAVA_HOME= ./gradlew :obs-agent-core:shadowJar
```

Produces `obs-agent-core/build/libs/obs-agent-core-0.1.0-all.jar` (shaded + relocated ByteBuddy/Jackson; slf4j uses the host's). Use the `-all` jar (has the manifest + deps); not the plain one.

> JDK 21 (GraalVM 21.0.11) builds Java 17 bytecode. The system default `java` is JDK 8 — always set `JAVA_HOME` to JDK 21 for `./gradlew`.

### 2. Attach to a running service (dynamic, no restart)

```bash
# find the service PID (e.g. on port 8080)
PID=$(netstat -ano | grep LISTENING | grep ":8080" | head -1 | awk '{print $NF}')

# attach the agent into the running JVM (Arthas-style)
JAVA_HOME=
JAR="/obs-agent-core/build/libs/obs-agent-core-0.1.0-all.jar"
ARGS='port=9911,token=tok,enableInstrumentation=true,chat.apiKey=,chat.base_url=,chat.model='
"$JAVA_HOME/bin/java" --add-modules jdk.attach -cp scripts Attach "$PID" "$JAR" "$ARGS"
```

The agent starts an MCP server + (optional) chat UI on port `9911` inside the target JVM. It stays until the JVM restarts.

> The `Attach` helper is a one-liner using `com.sun.tools.attach.VirtualMachine.loadAgent` (the `agentmain` entry point). It ships at `scripts/Attach.java`. To use it, compile once: `"$JAVA_HOME/bin/javac" --add-modules jdk.attach -d scripts scripts/Attach.java`.

### 2b. Or attach at JVM startup (`-javaagent`, persistent across restarts)

```bash
java -javaagent:/obs-agent-core/build/libs/obs-agent-core-0.1.0-all.jar=port=9911,token=tok,enableInstrumentation=true -jar your-app.jar
```

Use this if you want the agent resident from boot (survives restarts). Dynamic attach (above) is for ad-hoc diagnosis without restart.

### 3. Use it

**Chat web UI** (browser): open `http://localhost:9911/chat?token=tok` — a chat box; type questions, the LLM calls the tools and answers.

**Claude Code** (MCP client):
```bash
claude mcp add --transport http obs-agent http://localhost:9911/mcp --header "Authorization: Bearer tok"
# then chat: "which route is slowest? any recent errors?"
```

**curl** (manual tool call):
```bash
curl -X POST http://localhost:9911/mcp \
  -H "Authorization: Bearer tok" -H "Content-Type: application/json" \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"recent_metrics","arguments":{"route":"/api/x"}}}'
```

> All endpoints require `Authorization: Bearer `. The chat **UI page** also accepts `?token=` (browsers can't send headers on URL navigation).

---

## Configuration

All config is in the agent args (comma-separated `key=value`), optionally overridden by env vars (`OBSAGENT_PORT`, `OBSAGENT_HOST`, `OBSAGENT_TOKEN`).

| Key | Default | Description |
|---|---|---|
| `port` | 9000 | MCP + chat HTTP port |
| `host` | `127.0.0.1` | Network interface to bind. Defaults to loopback (**not** exposed). Set `0.0.0.0` only behind a firewall/reverse proxy **and** with `token` set. |
| `token` | (none → no auth) | MCP/chat Bearer token. **Set it** if the port is exposed beyond localhost. |
| `window` | 300s | Metrics/log retention window |
| `bucket` | 1s | Metrics time bucket |
| `slowThreshold` | 500ms | Slow-call threshold |
| `logBuffer` | 2000 | Log ring buffer size |
| `enableInstrumentation` | false | Master switch for `watch_method`/`trace_request` (instrumentation tier) |
| `watchLimit` | 10 | Max captures per `watch_method` |
| `watchTtl` | 60s | `watch_method` auto-uninstall TTL |
| `scrubPatterns` | (none) | Regexes (semicolon-separated) to scrub from tool results (tokens/PII) |
| `chat.apiKey` | (none → chat off) | LLM API key. Set to enable the built-in chat. |
| `chat.base_url` | — | OpenAI-compatible base URL (e.g. `https://api.openai.com/v1`, `https://open.bigmodel.cn/api/paas/v4`, `https://ark.cn-beijing.volces.com/api/coding/v3`) |
| `chat.model` | — | Model name (e.g. `gpt-4o-mini`, `glm-4`, `glm-5.2`) |

When `chat.apiKey` is unset, only MCP is exposed (chat UI returns 404).

---

## Tools

**Read-only** (always available):

| Tool | What it does |
|---|---|
| `get_routes` | Known routes + health (count, error rate, p95) |
| `recent_metrics(route?, window_seconds)` | Per-route count / p50/p95/p99 / error rate / qps |
| `slow_calls(top)` | Slowest recent requests (with traceId if host sets MDC) |
| `recent_errors(top)` | Recent error requests |
| `search_logs(query, level?, traceId?)` | Search the in-JVM log ring (host Logback) |
| `thread_dump(top, by=cpu\|blocked)` | Top CPU threads + deadlocks |

**Instrumentation tier** (gated by `enableInstrumentation=true`, default off):

| Tool | What it does |
|---|---|
| `watch_method(class, method, limit?, ttl_seconds?)` | **Start** capturing a method's invocations (args/return/duration/exception). Returns immediately (two-phase). |
| `watch_results(class, method)` | **Read** the captures from a prior `watch_method`. |
| `trace_request(class_pattern, ttl_seconds?, limit?)` | **Start** a method call-tree trace (Arthas-`trace`) for classes matching the regex. Returns immediately. |
| `trace_results()` | **Read** the call trees from a prior `trace_request`. |

**Two-phase design**: `watch_method`/`trace_request` install + return immediately; the trace/watch stays active for `ttl_seconds`. Generate traffic (hit the endpoint, or rely on frontend traffic), then call `watch_results`/`trace_results` to read. This is critical — a single blocking call would prevent the caller (LLM) from generating traffic during the window (→ empty results).

### trace_request usage

- `class_pattern` is a **regex**. Use the FQCN for specific classes.
- To see **cross-class call trees** (controller → service), trace **both** classes with `|`: e.g. `class_pattern="interview.guide.modules.x.KnowledgeBaseController|interview.guide.modules.x.service.KnowledgeBaseListService"`.
- Repeated `trace_request` calls **reset** the prior trace before installing the new one (no advice stacking).
- Example flow (via chat or MCP):
  1. `trace_request("...KnowledgeBaseController|...KnowledgeBaseListService", ttl_seconds=15)`
  2. hit the endpoint a few times
  3. `trace_results()` → tree: `controller.getStatistics (335ms) → service.getStatistics (333ms)`

---

## Architecture

```
┌──────────────────────────────────────────────────────────┐
│  LLM clients (BYO): chat UI (/chat) / Claude Code / curl  │
└──────────────────┬───────────────────────────────────────┘
                   │ MCP (JSON-RPC over HTTP) + /chat (POST), token auth
┌──────────────────▼───────────────────────────────────────┐
│  obs-agent-core  (inside the target JVM, shaded jar)      │
│  ┌───────────┐  ┌────────────┐  ┌─────────────────────┐   │
│  │ MCP server│  │ ChatEndpoint│  │  ToolRegistry       │   │
│  │ (JdkHttp) │  │ (LlmClient, │  │ (10 tools)          │   │
│  │           │  │  agent loop)│  │                     │   │
│  └───────────┘  └────────────┘  └──────────┬──────────┘   │
│  ┌──────────────────────────────────────────▼──────────┐ │
│  │ Source SPI (extension seam):                        │ │
│  │ MetricsSource / LogSource / TraceSource /            │ │
│  │ TopologySource  (+ capability flags)                 │ │
│  └──────────────────────────┬───────────────────────────┘ │
│  ┌──────────────────────────▼──────────────────────────┐│
│  │ Phase 1 in-JVM impls: ring buffers / Logback / JMX   ││
│  └──────────────────────────────────────────────────────┘│
│  ┌──────────────────────────────────────────────────────┐│
│  │ ByteBuddy instrumentation (shaded + isolated):        ││
│  │ - ServletMetricsInstrumentation (javax+jakarta)       ││
│  │ - MethodInstrumenter (watch)  - MethodTracer (trace)   ││
│  └──────────────────────────────────────────────────────┘│
└────────────────────────────────────────────┬──────────────┘
              (future) ┌──────────────────────▼──────────────┐
                       │ Phase 3: SkyWalkingTraceSource etc.  │
                       │ (query OAP, zero-change to tools)    │
                       └──────────────────────────────────────┘
```

**Key design points:**

- **Single shaded jar** — ByteBuddy/Jackson relocated to `io.obsagent.shaded.*`; slf4j is `compileOnly` (uses the host's, so the agent can read host Logback/MDC). No host-classpath pollution.
- **No new deps for chat** — uses JDK `java.net.http.HttpClient` (OpenAI-compatible).
- **Advice never throws into user code** — all ByteBuddy advice is wrapped in try/catch + JUL logging.
- **Advice-accessed members are `public static`** — ByteBuddy inlines advice into target classes in *other packages*; package-private fields/methods would `IllegalAccessError` (a real gotcha — see Troubleshooting).
- **`agentmain`** — enables dynamic attach to a running JVM (not just `-javaagent` at startup).

---

## Roadmap

- ✅ **Phase 1** — core agent: servlet metrics + ring buffers + Source SPI + in-JVM sources + MCP server + read-only tools + scrubber/guard + shaded jar + e2e smoke test.
- ✅ **Phase 2** — `watch_method` (method capture) + `trace_request` (call-tree) + validation demo.
- ✅ **Built-in chat** — LlmClient (OpenAI-compatible) + ChatEndpoint (agent loop) + web UI at `/chat`.
- ✅ **Dynamic attach** (`agentmain`) + two-phase trace/watch + trace reset (no stacking) + cross-class call trees.
- ⏳ **Phase 3** (deferred) — `SkyWalkingTraceSource`/`SkyWalkingTopologySource` (query OAP) → `distributed_trace`/`service_topology` tools → cross-service business-chain root cause (order→pay). The Source SPI is already in place.
- ⏳ **Optional tail** — Spring Boot adapter (route templates/Actuator/Micrometer traceId); streaming chat (token-level SSE); `obs-agent-cli`; reactive/WebFlux; Java 8 bytecode broadening.

---

## Known limitations

- **`trace_request` is pattern-based, not traceId-based** — the host must set MDC `traceId` (e.g. Micrometer Tracing) for `slow_calls`/`search_logs` to correlate by traceId. Without it, those fields are `null` (route+latency diagnosis still works).
- **Broad `.*` patterns** capture the entry method but may not show cross-class callees as children — trace the **specific classes** (with `|`) for cross-class trees.
- **Chat is non-streaming** (POST `/chat/completions`, `stream=false`) — returns the full answer after the tool-calling loop. Live token streaming is a follow-up.
- **Servlet-only** (blocking) — reactive/WebFlux not yet instrumented.
- **Java 17 bytecode** — broadening to Java 8 (attach to older JVMs) is a follow-up.
- **Agent can't be hot-updated in a running JVM** — re-attaching a new jar doesn't redefine already-loaded classes (system classloader caches them). To pick up code changes, restart the target JVM + re-attach (or use `-javaagent` at startup).
- **`watch_method`/`trace_request` default OFF** — set `enableInstrumentation=true` to use them.

---

## Troubleshooting

**MCP client (Claude Code) fails to connect with "expected object, received undefined" for `capabilities`/`serverInfo`** — the `initialize` result must include both (MCP spec). Fixed (returns `{"protocolVersion","capabilities":{"tools":{}},"serverInfo":{"name":"obs-agent","version":"0.1.0"}}`).

**MCP tools return "content expected array, received object" / content blocks fail validation** — `tools/call` result must be `{"content":[{"type":"text","text":""}],"isError":false}` (array of content blocks, each with a `type`). Fixed.

**`search_logs` returns "no log source" / `traceId` always null** — slf4j must NOT be shaded (so the agent uses the host's slf4j to reach Logback + MDC). slf4j is `compileOnly` + not relocated. If you see this, check the shadow config.

**Browser opens `/chat` → `{"code":-32001,"message":"unauthorized"}`** — browsers can't send `Authorization` headers on URL navigation. Open `http://localhost:9911/chat?token=tok` (query param accepted).

**`trace_request` returns empty `{"trees":[]}`** — (1) it's **two-phase**: call `trace_request` (install), generate traffic, then `trace_results` (read). The install call returns immediately, it does NOT capture. (2) Make sure `enableInstrumentation=true`. (3) Verify the `class_pattern` regex matches (use the FQCN; `.` matches any char, `*` is a quantifier).

**`trace_request` shows duplicate/nested trees** — fixed: repeated `trace_request` now `reset()`s the prior transformer before installing. If you still see it, you're on an old jar (rebuild + re-attach).

**ByteBuddy advice silently does nothing (empty captures, no error)** — the advice accesses package-private members from a cross-package target → `IllegalAccessError` swallowed by the try/catch → returns null. Advice logic must be in `public static` methods (e.g. `onEnter`/`onExit`) so the inlined advice only calls public methods. The unit test passes (same-package test class) but live cross-package targets fail — always verify on a fresh JVM with a real cross-package target.

**Build fails with JDK issues** — system `java` is JDK 8; the project needs JDK 21. Always run `./gradlew` with `JAVA_HOME=`.

**Re-attaching a second agent doesn't apply new code** — the first attach's classes are already loaded by the system classloader; `loadAgent` won't redefine them. Restart the target JVM + attach once.

---

## Development

```bash
# build
JAVA_HOME="..." ./gradlew :obs-agent-core:build
# tests
JAVA_HOME="..." ./gradlew :obs-agent-core:test
# e2e smoke (real -javaagent attach + MCP)
JAVA_HOME="..." ./gradlew :obs-agent-core:test --tests 'io.obsagent.EndToEndSmokeTest' -Pe2e=true
# shaded agent jar
JAVA_HOME="..." ./gradlew :obs-agent-core:shadowJar
```

Layout:
- `obs-agent-core/src/main/java/io/obsagent/` — `AgentPremain`, `config/`, `data/` (ring buffers + `model` records), `source/` (SPI + `injvm/` impls), `instr/` (ByteBuddy: servlet/watch/trace), `mcp/` (JdkHttpMcpServer), `tools/` (ReadOnlyTools + InstrumentingTools), `safety/` (Scrubber + Guard), `chat/` (LlmClient + ChatEndpoint + ChatPage).
- `docs/superpowers/specs/` — design spec.
- `docs/superpowers/plans/` — Phase 1 / Phase 2 / chat+trace implementation plans.
- `.superpowers/sdd/progress.md` — SDD execution ledger (gitignored).

---

## How it was built

Designed via brainstorming → spec → implementation plans → subagent-driven development (12 Phase-1 tasks + 4 Phase-2/chat-trace tasks),

…

## Source & license

This open-source MCP server is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [twz007](https://github.com/twz007)
- **Source:** [twz007/java-obs-agent](https://github.com/twz007/java-obs-agent)
- **License:** Apache-2.0

Install and usage instructions live in the source repository linked above.

## Pricing

- **Free** — Free

## Security capabilities

Automated source analysis of v0.1.0 — what this tool can access:

- **Network access:** yes
- **Filesystem access:** no
- **Shell / process execution:** no
- **Environment & secrets:** no
- **Dynamic code execution:** no

*"Yes" means the capability is present in the source — more access means more to trust, not that it is unsafe.*


## Versions

- **0.1.0** — security scan: passed — Imported from the upstream source.

## Links

- Listing page: https://agentstack.voostack.com/l/mcp-twz007-java-obs-agent
- Seller: https://agentstack.voostack.com/s/twz007
- Browse the marketplace: https://agentstack.voostack.com/browse

---
Listed on AgentStack — the marketplace for AI agent skills and MCP servers. Every listing is security-reviewed. Creators keep 70%.