Log Analyzer

Log Analyzer

Purpose

Use this skill to extract actionable, evidence-backed findings from logs — not to dump filtered output. The analysis must:

• detect log format before parsing,
• redact PII before quoting,
• distinguish first error from root cause,
• separate noise from signal with statistical reasoning,
• correlate across services using trace_id / request_id,
• and end with a prioritised list of hypotheses + recommendations.

The skill is the upstream half of incident response: it produces the evidence package that incident-postmortem formats into a blameless RCA. Use them together, in that order.

Quick Reference

| When you need to… | Jump to | |---|---| | Pick analysis depth (Lite / Standard / Strict) | §Analysis Modes | | Detect what kind of logs you are looking at | §Mandatory Gates → Gate 1 | | Avoid leaking secrets / PII in the report | §Mandatory Gates → Gate 2 | | State the time window you analysed | §Mandatory Gates → Gate 3 | | Decide whether N errors is signal or noise | §Mandatory Gates → Gate 4 | | Correlate logs across services | §Mandatory Gates → Gate 5 | | Know what NOT to call a root cause | §Mandatory Gates → Gate 6 | | Cap finding volume, route overflow to Residual Risk | §Mandatory Gates → Gate 7 | | Execute the analysis end-to-end | §Workflow | | See a complete formatted output | Load references/example-output.md | | Hand off to a post-mortem | §Hand-off Protocol |

When To Use

Trigger this skill when:

• the user asks to analyse / investigate / look at / triage logs,
• the user pastes log lines, a file path, a kubectl logs / journalctl dump, or a Loki/ELK/Datadog query result,
• the user reports an outage / regression / flaky behaviour and points at log evidence,
• another skill needs an evidence package before producing a post-mortem (incident-postmortem) or a runbook update.

Do not use this skill for:

• writing new log statements in source code (that is go-observability-review territory),
• designing alerts / dashboards (monitoring-alerting),
• formal post-mortem authoring (incident-postmortem),
• debugging code without log evidence (systematic-debugging).

Analysis Modes (Lite / Standard / Strict)

State the chosen mode in the report. Default: Standard.

Mode selection rules:

• Choose Lite only when scope is small (single service, ≤ 1 hour window, or HOST PROC[PID]: | syslog | awk, grep, journalctl -p` |

| Lines start with ISO-8601 + level keyword (INFO, ERROR) | text-structured | grep, awk | | Output of kubectl logs ... (optional -c) | container stdout | kubectl logs --since=… --tail=…, then treat per inner format | | Output of journalctl -o json | systemd journal JSON | jq, journalctl --since … --until … | | Multi-line stack traces (Go panic, Java exception, Python traceback) | mixed — needs aggregation | block-aware awk / rg --multiline |

Record in Execution Status: Format: plus the regex / jq filter the analysis used. If multiple formats are present (e.g., a JSON service log that includes a panic stack trace), state both and split the analysis.

2) PII / Secret Redaction Gate

Logs frequently contain credentials, tokens, customer identifiers, and personal data. The report MUST NOT echo unredacted secrets back to the user. This is a hard requirement, not advisory.

Always redact before quoting:

| Class | Examples | Redaction | |---|---|---| | Bearer tokens / JWTs | Authorization: Bearer eyJ…, Bearer eyJhbGc… | Bearer ***REDACTED*** | | API keys | sk-…, AKIA…, xoxb-…, ghp_… | ***REDACTED-API-KEY*** | | Passwords / secrets in URLs | postgres://user:hunter2@host/db | postgres://user:***@host/db | | Email addresses | alice@example.com | a***@example.com | | Phone numbers | +1-555-123-4567 | +1-***-***-4567 | | Credit card / IBAN | 13–19 digit groups | ***REDACTED-PAN*** | | Government IDs | SSN 123-45-6789, others | ***REDACTED-ID*** | | Cookies / session IDs | Cookie: session=… | Cookie: session=*** |

Quoting rule: when including a sample log line in the report, present it as a fenced code block with the redaction already applied. Never paste raw lines verbatim if they contain any of the above.

If you are unsure whether a field is sensitive, redact by default and note (redacted by analyst — uncertainty) next to the line.

3) Time Window Boundary Gate

Every report MUST explicitly state the analysed time window in absolute UTC, with the source of those bounds. This prevents the most common error in log review: drawing conclusions from an unrepresentative slice.

Required record:

• Window: → (ISO-8601, e.g., 2026-04-28T08:14:00Z → 2026-04-28T09:30:00Z),
• Source: ,
• Coverage: . State partial when log rotation, retention, sampling, or a paged-out aggregator query may have truncated the data.

If the user only provides a snippet without timestamps, state Window: unknown — only N lines provided, no timestamp parse possible and stop drawing time-based conclusions.

4) Statistical Significance Gate

A raw count is meaningless without a denominator. Before calling something "frequent" or "spike":

• compute or estimate the request / event base rate for the same window (e.g., total request volume, healthy traffic count),
• compare against a baseline window of the same length when available (e.g., the previous hour, same hour yesterday),
• prefer rate (errors/sec, error_ratio = errors / total) over absolute counts.

Heuristic anchors (state when you apply them):

| Pattern | Likely signal? | |---|---| | error_ratio ≥ 1% sustained over the window | Yes | | error_ratio `** rather than presenting it as confirmed.

7) Volume Cap & Severity-Tiered Reporting Gate

Findings have a soft cap by mode (5 / 10 / 15). Everything above the cap goes to Residual Risk / Investigation Gaps rather than being silently dropped.

Phases:

• Phase 1 — High: Report ALL High findings. High is never dropped by the cap.
• Phase 2 — Medium: Fill remaining slots with Medium findings, ordered: customer-impacting → engineer-debug-blocker → operational hygiene.
• Phase 3 — Low: Only if slots remain.
• Overflow: Move displaced candidates to Residual Risk with one-line summary (severity | category | location | one-line description) and add N additional issues moved to Residual Risk to the Summary.

Example: Standard mode, 3 High + 9 Medium found → report 3 High + 7 Medium as findings, move 2 Medium to Residual Risk.

Workflow

Steps run in order. Skip a step only if it does not apply, and state why in Execution Status.

1. Select mode (Lite | Standard | Strict) and record selection rationale.

1. Define scope.

• Confirm log sources: file paths, glob patterns, kubectl / journalctl arguments, aggregator query strings.
• Estimate volume (wc -l, du -sh, or query result count). For files > 1 GB or > 10 M lines, refuse line-by-line scanning and switch to streaming patterns (see references/log-tooling-commands.md).
• State the hypothesis the user gave you. ("Why are checkout 502s spiking" is different from "Show me everything that broke today.")

1. Apply Format Detection Gate (Gate 1). Record the detected format and parsing tooling.

1. Apply PII / Secret Redaction Gate (Gate 2) before opening files for direct quoting. Decide the redaction set up-front — do not retro-redact after writing the report.

1. Apply Time Window Gate (Gate 3). Lock the bounds. If the user implies "today" or "recently", convert to absolute UTC and state it.

1. Reference Loading Gate. Load reference files matching the situation before drawing conclusions:

• JSON / structured logs → references/log-format-cheatsheet.md
• Multi-service / microservices → references/log-correlation.md
• Loki / ELK / Datadog / CloudWatch query needed → references/log-aggregator-queries.md
• "Is this a real spike?" — references/log-statistical-methods.md
• About to quote a log line → references/log-pii-redaction.md
• Cascading failures suspected → references/log-cascade-analysis.md
• Tooling refresher (jq / awk / kubectl / journalctl) → references/log-tooling-commands.md
• Always loaded → references/log-anti-patterns.md, references/log-analysis-quick-checklist.md

Record loaded references in Execution Status. This is a mandatory gate: drawing conclusions about correlation, statistical significance, or PII handling without loading the matching reference is a contract violation.

1. Quick scan. Counts by level / class. Top error patterns. First and last occurrence per class. Compute error_ratio if denominator data is available.

1. First-occurrence pivot. For each High-severity error class, locate the first instance in the window and capture the 30 lines of context before it (the run-up, not just the failure). The cause usually lives there, not at the first ERROR line.

1. Apply Correlation Gate (Gate 5). For ≤ 3 representative failed requests / traces, walk the full lifecycle across services. Build a per-request timeline.

1. Cascade analysis. Cluster errors that share a trigger (e.g., 12 downstream timeouts that all hit at +200ms after one upstream failure). Distinguish cause cluster vs symptom cluster. Reference: log-cascade-analysis.md.

1. Apply Statistical Significance Gate (Gate 4) on every "frequent" / "spike" claim. Downgrade or drop claims that fail it.

1. Apply Causation Discipline Gate (Gate 6). Construct the causation chain for each leading hypothesis: symptom → proximate trigger → underlying cause → contributing factors.

1. Hypotheses & severity. Produce ≥ 2 root cause hypotheses ranked by likelihood, each grounded in quoted evidence. State what additional data would confirm or refute each.

1. Apply Volume Cap Gate (Gate 7). Tier findings, route overflow to Residual Risk.

1. Hand-off. If the user is heading to a post-mortem, fill the Hand-off Protocol section with the structured fields incident-postmortem consumes (see §Hand-off Protocol).

Severity Rubric

• High: customer-visible outage / data loss / data corruption / security event / SLO breach actively in progress / debugging fundamentally degraded (e.g., correlation IDs absent in production logs).
• Medium: latent reliability or maintainability defect, error class that increases on-call toil but has not yet caused user impact, observability gap that masks a category of failures.
• Low: log hygiene issue, cosmetic noise, redundant fields, format inconsistency that is annoying but not load-bearing.

Evidence Rules

• Every finding MUST include:
• exact source location (path:line, or aggregator query + timestamp range, or kubectl logs -c + window),
• the redacted log sample (1–5 lines),
• the inference made from that sample,
• what would refute the inference.
• Clearly label Confirmed vs Hypothesis vs Hypothesis — needs corroboration. Promoting a hypothesis to confirmed without evidence is a contract violation.
• Do not fabricate timestamps, IDs, or counts. If a number is approximate, write ~ and explain.

Anti-patterns (DO NOT report these as findings)

See references/log-anti-patterns.md (always loaded). Examples:

• Treating the first ERROR line as the root cause without checking the run-up.
• Calling 3 errors in 100 M log lines a "spike".
• Quoting a log line containing Bearer eyJhbGc… without redaction.
• Reporting level=warn lines from a healthy retry path as defects.
• Drawing conclusions from a 30-second slice of a 12-hour incident.

Output Format (Required)

Analysis Mode

• Lite | Standard | Strict
• mode selection rationale (1–2 lines)

Window & Source

• Window: →
• Source:
• Coverage: full | partial —
• Format: (per source if mixed)

Executive Summary

1–3 lines. Lead with the answer to the user's actual question. Include origin breakdown: X confirmed / Y hypothesis / Z needs corroboration. If volume cap fired, note: N additional issues moved to Residual Risk.

Findings

List findings ordered by severity (High → Medium → Low), then by confidence (Confirmed → Hypothesis).

[High|Medium|Low] Short title

• ID: LOG-001
• Confidence: Confirmed | Hypothesis | Hypothesis — needs corroboration
• Category: availability | latency | data-integrity | security | observability | hygiene
• Location: path:line or ` or @ ` (or location list for merged findings)
• Evidence: redacted log sample (fenced code block, ≤ 5 lines)
• Inference: what the evidence implies
• Causation chain (when applicable): symptom → proximate → underlying → contributing
• Refuter: what additional data would prove this wrong
• Recommendation: specific next action

Timeline

Chronological reconstruction of the most-impactful failed flow(s), one row per significant event. Use UTC timestamps.

| Time (UTC) | Service | Event | traceid / requestid | |---|---|---|---|

Correlation Map

For Standard / Strict modes when multiple services are involved. List the failed traces walked end-to-end with cross-service hops, statuses, and latencies. If correlation IDs are absent, state Correlation IDs missing — see High Observability finding.

Root Cause Hypotheses

Ranked, each with:

• the chain (symptom → underlying),
• supporting evidence,
• refuter,
• next data needed to confirm.

Recommendations

Numbered, ordered by impact × ease. For each: Owner suggestion, Effort (S/M/L), and Expected effect.

Suppressed Items

Patterns that looked alarming but were suppressed by the gates. One line each: pattern + why suppressed (base rate / known healthy retry / non-user-controlled / …).

Execution Status

• Format: detected per source
• Window: as above
• Files / queries scanned: count + total size
• References loaded: list
• PII redaction applied: yes (categories: …) | no — none detected
• Statistical baseline: | unavailable — reason
• Correlation IDs present: trace_id | request_id | span_id | user_id (✓/✗)
• External tools run: jq | rg | awk | kubectl logs | journalctl | aggregator query with PASS / FAIL / Not run + reason
• If a tool was not run, state Not run in this environment plus the exact command the user can run.

Open Questions

Only blockers that materially change the conclusion (e.g., "do you have logs from the upstream gateway? if not, my hypothesis 2 cannot be ruled out").

Residual Risk / Investigation Gaps

• Volume-cap overflow (severity | category | location | one-line description) — so no validated issue is silently dropped.
• Time-window gaps — log rotation / retention / sampling that may have hidden evidence.
• Coverage gaps — services or hops you could not see.

Hand-off Protocol

Fill this section when the user is heading to a post-mortem (always for Strict mode, and on request for Standard). The fields map directly into incident-postmortem Gate 1.

incident_id: 
impact_summary: 
window_utc:  → 
affected_services: 
data_sources: 
top_findings: [LOG-001, LOG-002, …]   # IDs from §Findings
leading_hypothesis: 
blameless_framing:    # avoid naming individuals

Summary

1–3 lines. Restate the leading hypothesis, the confidence level, and the next concrete data the user should fetch. If volume cap fired, include N additional issues moved to Residual Risk.

No-Finding Case

If the logs in scope show no actionable issue:

• Explicitly say: No actionable findings in window.
• Still produce: Analysis Mode, Window & Source, Executive Summary (one line: "Window appears healthy."), Execution Status, Statistical Significance note (error_ratio for the window), Coverage gaps (anything you could not see), Recommendations (e.g., widen window, add missing correlation ID).

A "no findings" report with no Execution Status sect

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: johnqtcg
• Source: johnqtcg/awesome-skills
• License: MIT
• Homepage: https://johnqtcg.github.io/awesome-skills/

Install and usage instructions live in the source repository linked above.