Instrument Observability

Instrument A365 Observability

> Trigger phrases — any of these will activate this skill automatically: > - "instrument observability for this agent" > - "add a365 observability to this agent" > - "add observability to this agent" > - "set up tracing for this agent" > - "make this agent visible in microsoft defender" > - "enable agent 365 telemetry" > - "wire up opentelemetry for this agent" > - "add observability to this .net agent" > - "add observability to this node.js agent" > - "add a365 observability to this python agent"

---

Overview

This skill instruments Microsoft Agent 365 observability into an existing agent codebase without disrupting the agent's core logic. It:

1. Detects the agent type (.NET AgentFramework, Node.js, or Python)
2. Installs the correct A365 observability packages (core + hosting + optional extensions)
3. Wires observability in the entry point
4. Adds BaggageBuilder context or BaggageMiddleware to message handlers
5. Implements the agentic token resolver with caching
6. Adds manual instrumentation scopes (InvokeAgentScope, InferenceScope, ExecuteToolScope — required for store publishing)
7. Updates configuration files with observability settings
8. Validates the build passes

> Store publishing requirement: The Agent 365 store validation requires InvokeAgentScope, > InferenceScope, and ExecuteToolScope to be implemented. This skill wires them.

All changes are additive and idempotent — rerunning the skill is safe.

---

Phase 0: Load Detection Cache and Validate

> Task-list display (applies throughout this skill). This skill creates tasks inline via **TaskCreate** — "..." markers at the start of each phase, and marks them complete at phase end. The user must see this progress visibly. Each TaskCreate line corresponds to one checklist item; exactly one item in_progress at a time. > - Claude Code: TaskCreate is in allowed-tools — calling it renders a native checklist UI; subsequent TaskUpdate calls flip statuses. > - VS Code Copilot Chat / GitHub Copilot CLI: allowed-tools is ignored — before Phase 0.1, scan this SKILL.md for all **TaskCreate** — "..." lines and emit a markdown checklist in chat upfront (- [ ] Load detection cache…, - [ ] Determine agent kind…, etc.); flip items to - [x] as each phase completes.

TaskCreate — "Load detection cache and validate with user"

Step 0.1 — Triage the workspace

Run in parallel:

• Glob **/*.csproj, package.json, requirements.txt, pyproject.toml, src/**/*.ts, **/*.cs, **/*.py → hasProjectFiles.
• Read .a365-workspace-detection.local.json → cacheState (fresh if detectedAt doesn't patch your LLM library directly. The skill will still wire useMicrosoftOpenTelemetry/UseMicrosoftOpenTelemetry (OTel SDK + A365 exporter), but you'll need to manually wrap each LLM call with InferenceScope.start(...) to capture gen_ai. spans. See -observability.md` § 'InferenceScope — Manual Wrapping' for the pattern." Continue to Phase 2.

1. TaskUpdate — Mark complete and report detected agent type (+ any soft-warn) to user.

---

Phase 2: Install A365 Observability Packages

TaskCreate — "Install A365 observability packages"

All languages converge on a single unified distro that re-exports the legacy A365 observability + hosting types and auto-instruments common LLM SDKs:

| Language | Install command | S2S extra (FMI token chain) | |----------|-----------------|------------------------------| | .NET | dotnet add package Microsoft.OpenTelemetry | dotnet add package Azure.Identity Microsoft.Identity.Client | | Node.js | npm install @microsoft/opentelemetry | npm install @azure/msal-node @azure/identity | | Python | pip install microsoft-opentelemetry | pip install msal azure-identity httpx |

Do not install legacy *.Observability.Runtime / -hosting / -extensions-* packages alongside the unified distro — the distro re-exports their types and mixing the two produces CS0433 duplicate-type errors (.NET) or duplicate spans (Node.js / Python). After install, verify the package appears in the manifest (*.csproj / package.json / requirements.txt or pyproject.toml). pip install does not update the dependency manifest — prefer uv add microsoft-opentelemetry (or poetry add ...) for Python.

Python — Google ADK gotcha: if pyproject.toml lists google-adk, uv sync will backtrack for minutes resolving the OTel graph. Pin OTel via [tool.uv] override-dependencies — see python-observability.md → "Google ADK projects — pin the OTel stack" for the exact block. Other Python stacks (AgentFramework, LangChain, OpenAI, Claude, Semantic Kernel) don't need this.

Full per-language package tables, version constraints, and the LangChain extras flag live in the references — see the "Required packages" section of:

• ${CLAUDE_PLUGIN_ROOT}/skills/instrument-observability/references/dotnet-observability.md
• ${CLAUDE_PLUGIN_ROOT}/skills/instrument-observability/references/nodejs-observability.md
• ${CLAUDE_PLUGIN_ROOT}/skills/instrument-observability/references/python-observability.md

TaskUpdate — Mark complete.

---

Phase 3: Wire Observability in Entry Point

TaskCreate — "Wire observability in entry point"

> Pre-existing placeholders: As of CLI 1.1, a365 setup all auto-writes Agent365Observability placeholder sections to appsettings.json (.NET) or .env (Node.js/Python). Before creating config from scratch, check if placeholders already exist and fill in values rather than duplicating the section.

For .NET AgentFramework

1. Read the current entry point (Program.cs or detected file).

1. Edit — Add observability wiring following the reference pattern in dotnet-observability.md:

• Add using Microsoft.OpenTelemetry; to Program.cs.
• OBO / agentic-user path (AI Teammate AND Standard .NET agents): Call builder.UseMicrosoftOpenTelemetry(o => { ... }) with o.Exporters = ExportTarget.Agent365 | ExportTarget.Console (Dev) or ExportTarget.Agent365 (Production). The distro auto-registers IExporterTokenCache in DI — no AddAgenticTracingExporter() call needed. Leave o.Agent365.Exporter.UseS2SEndpoint at its default (false) — the exporter POSTs to /observability/ which the OBO token cache authenticates. Also set "EnableAgent365Exporter": true in appsettings.json to activate the backend exporter — the SDK defaults this to false when absent, so without it the exporter is wired but inert.
• Required: IChatClient.UseOpenTelemetry() — when registering the IChatClient (e.g. Azure OpenAI), chain .AsBuilder().UseFunctionInvocation().UseOpenTelemetry(sourceName: null, cfg => cfg.EnableSensitiveData = true).Build(). This is what makes the AI SDK emit the gen_ai.inference and gen_ai.tool spans that InvokeAgentScope (Phase 5.5) anchors as children. Skipping this means no LLM spans appear in MAC, even with everything else wired — the InvokeAgent parent becomes a hollow span. EnableSensitiveData = true includes prompts/completions in span attributes (PII consideration — set to false for regulated data).
• S2S path: First Write the two scaffold files from the reference doc — Observability/ObservabilityServiceExtensions.cs (DI extension with AddAgent365Observability() using ServiceTokenCache and conditional ObservabilityTokenService) and Observability/ObservabilityTokenService.cs (background service that acquires the Observability API token via the MSAL FMI 3-hop chain with .WithFmiPath() targeting scope api://9b975845-388f-4429-889e-eab1ef63949c/.default, supports MSI with client-secret fallback). Then call builder.Services.AddAgent365Observability(); and builder.UseMicrosoftOpenTelemetry(...) with token resolver reading from the ServiceTokenCache. Critical: Set o.Agent365.Exporter.UseS2SEndpoint = true in the options callback — without this, the exporter posts to the wrong path (/observability/ instead of /observabilityService/) and gets HTTP 401. See "Known Issues" section.
• Optionally register adapter.Use(new BaggageTurnMiddleware()) (OBO path only) to auto-populate baggage on every request
• Mark all new lines with: // A365 Observability — best-effort instrumentation (verify against official sample)

1. Preserve all existing code — only add new lines, never remove.

For Node.js

1. Read the current entry point (index.ts, app.ts, or detected file).

1. Edit — Add observability initialization following the reference pattern in nodejs-observability.md:

• Import useMicrosoftOpenTelemetry, shutdownMicrosoftOpenTelemetry, configureA365Hosting, and AgenticTokenCacheInstance — all from @microsoft/opentelemetry (single package as of GA 1.0; do NOT import from the legacy -observability, -hosting, or -runtime packages).
• OBO / agentic-user path: Call useMicrosoftOpenTelemetry({ a365: { enabled: true, enableObservabilityExporter: true, tokenResolver } }) before any LLM/framework imports. Both enabled: true AND enableObservabilityExporter: true are required in 1.0+ to actually export spans. Wire tokenResolver to AgenticTokenCacheInstance.getObservabilityToken(agentId, tenantId) ?? ''.
• S2S path: First Write observability/token-cache.ts (in-memory token cache with cacheToken/getCachedToken/tokenResolver) and observability/observability-token-service.ts using the scaffold pattern from nodejs-observability.md (S2S section). This module acquires the Observability API token via MSAL FMI 3-hop chain (@azure/msal-node with fmiPath parameter, targeting scope api://9b975845-388f-4429-889e-eab1ef63949c/.default, supports MSI with client-secret fallback) and refreshes it every 50 min. Then call useMicrosoftOpenTelemetry({ a365: { enabled: true, enableObservabilityExporter: true, useS2SEndpoint: true, tokenResolver: a365TokenResolver } }). useS2SEndpoint: true is now a first-class option (1.0+); the old workaround with custom Agent365Exporter via spanProcessors and ENABLE_A365_OBSERVABILITY_EXPORTER=false is no longer needed for new instrumentation. If the old workaround is already present in an existing agent, leave it in place — do not delete code as part of this additive skill; flag it in the final summary as a candidate for cleanup if the user explicitly asks to migrate.
• Both paths: Call configureA365Hosting(adapter, { enableBaggage: true }) once at startup to register BaggageMiddleware. This replaces manual adapter.use(new BaggageMiddleware()) and removes the need for BaggageBuilderUtils.fromTurnContext in handlers.
• Both paths: Register SIGTERM/SIGINT handlers calling await shutdownMicrosoftOpenTelemetry() to flush pending spans on shutdown.
• Auto-instrumentation note: Do NOT call OpenAIAgentsTraceInstrumentor.enable() or LangChainTraceInstrumentor.instrument() — these are auto-enabled in 1.0+ and manual calls cause duplicate spans. To opt out, set instrumentationOptions: { openaiAgents: { enabled: false } }.
• Mark all new lines with: // A365 Observability — best-effort instrumentation (verify against official sample)

1. Preserve all existing code — only add new lines, never remove.

For Python

1. Read the current entry point (app.py, host_agent_server.py, or detected file).

1. Edit — Add observability configuration following the reference pattern in python-observability.md:

• Import use_microsoft_opentelemetry from microsoft.opentelemetry (single unified package; do NOT import from the legacy microsoft_agents_a365.* namespace).
• OBO / agentic-user path: Call use_microsoft_opentelemetry(enable_a365=True, a365_enable_observability_exporter=True, a365_token_resolver=...). Both enable_a365=True AND a365_enable_observability_exporter=True are required in 1.0+ to actually export spans. Wire a365_token_resolver to AgenticTokenCache().get_observability_token from microsoft.opentelemetry.a365.hosting.token_cache_helpers (or a custom resolver reading from token_cache.py).
• S2S path: First Write observability/token_cache.py (in-memory token cache with cache_token/get_cached_token) and observability/observability_token_service.py using the scaffold pattern from python-observability.md (S2S section). This module acquires the Observability API token via a 3-hop FMI chain: direct HTTP POST with fmi_path for Hops 1+2 (MSAL Python does not properly serialize fmi_path — known limitation), then msal.ConfidentialClientApplication for Hop 3, targeting scope api://9b975845-388f-4429-889e-eab1ef63949c/.default, supports MSI with client-secret fallback, refreshes every 50 min via an asyncio background task. Then call use_microsoft_opentelemetry(enable_a365=True, a365_enable_observability_exporter=True, a365_use_s2s_endpoint=True, a365_token_resolver=...). a365_use_s2s_endpoint=True is now a first-class kwarg — no workaround needed. Schedule run_token_service() as an asyncio task and call acquire_initial_token() in your aiohttp lifespan startup. Also install msal, azure-identity, and httpx.
• Both paths: Call ObservabilityHostingManager.configure(adapter.middleware_set, ObservabilityHostingOptions(enable_baggage=True)) once at startup to auto-populate baggage from TurnContext. Note: enable_baggage defaults to False — must be explicitly set to True.
• Auto-instrumentation note: Do NOT call legacy *Instrumentor().instrument() methods for LangChain/OpenAI/SK/AgentFramework — these are auto-enabled in 1.0+ and manual calls cause duplicate spans.
• Mark all new lines with: # A365 Observability — best-effort instrumentation (verify against official sample)

1. Preserve all existing code — only add new lines, never remove.

1. TaskUpdate — Mark complete.

---

Phase 4: Add BaggageBuilder Context to Message Handler

TaskCreate — "Add BaggageBuilder context to message handler"

> Skip this phase if BaggageMiddleware was registered in Phase 3 — the middleware handles > baggage propagation automatically for every request.

> Auth mode note: All three authMode values use authHandlerName: "AGENTIC" in the > code — the token exchange call is identical. The identity in traces is determined by Azure AD > provisioning and the incoming token. Add an inline comment indicating which mode was chosen.

For .NET AgentFramework

1. Read the detected message handler file.

1. Edit — Follow the reference pattern in dotnet-observability.md (full code sample under "Agent Class — Message Handler (OBO Path)"):

OBO path (obo / agentic-user) — applies to both AI Teammate agents and Standard .NET agents:

• Inject IExporterTokenCache in the constructor (auto-registered by the distro — no AddAgenticTracingExporter() call needed).
• Inject IConfiguration (for blueprint/observability config) and ILogger.
• Resolve agent ID for BOTH auth paths — agentic instance ID from the Activity for agentic turns, decoded from the auth token via Utility.ResolveAgentIdentity(context, authToken) for non-agentic turns (the SDK names the second parameter generically authToken — it accepts both OBO tokens and agentic-path tokens returned by UserAuthorization.GetTurnTokenAsync). Do NOT fall back to Guid.Empty.ToString() — that creates a synthetic identity the exporter cannot authenticate, polluting traces with "No token obtained. Skipping export for this identity." warnings.

```csharp string? resolvedAgentId = null; if (turnContext.Activity.IsAgenticRequest()) { resolvedAgentId = turnContext.Activity.GetAgenticInstanceId(); } else if (!string.IsNullOrEmpty(authHandlerName

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: microsoft
• Source: microsoft/agent365-skills
• License: MIT

Install and usage instructions live in the source repository linked above.