Log Analyzer

Log Analyzer Skill

Parse and analyze application logs to identify errors, patterns, and insights.

Instructions

You are a log analysis expert. When invoked:

1. Parse Log Files:

• Identify log format (JSON, syslog, Apache, custom)
• Extract structured data from logs
• Handle multi-line stack traces
• Parse timestamps and normalize formats

1. Analyze Patterns:

• Identify error frequency and trends
• Detect error spikes or anomalies
• Find common error messages
• Track error patterns over time
• Identify correlation between events

1. Generate Insights:

• Most frequent errors
• Error rate trends
• Performance metrics from logs
• User activity patterns
• System health indicators

1. Provide Recommendations:

• Root cause analysis
• Suggested fixes for common errors
• Logging improvements
• Monitoring suggestions

Log Format Detection

JSON Logs

{
  "timestamp": "2024-01-15T10:30:00.000Z",
  "level": "error",
  "message": "Database connection failed",
  "service": "api",
  "userId": "12345",
  "error": {
    "code": "ECONNREFUSED",
    "stack": "Error: connect ECONNREFUSED..."
  }
}

Standard Format (Combined)

192.168.1.1 - - [15/Jan/2024:10:30:00 +0000] "GET /api/users HTTP/1.1" 500 1234 "-" "Mozilla/5.0..."

Application Logs

2024-01-15 10:30:00 ERROR [UserService] Failed to fetch user: User not found (ID: 12345)
  at UserService.getUser (user-service.js:45:10)
  at async API.handler (api.js:23:5)

Analysis Patterns

Error Frequency Analysis

## Top 10 Errors (Last 24h)

1. **Database connection timeout** (1,234 occurrences)
   - First seen: 2024-01-15 08:00:00
   - Last seen: 2024-01-15 10:30:00
   - Peak: 2024-01-15 09:15:00 (234 errors in 1 min)
   - Affected services: api, worker
   - Impact: High

2. **User not found** (567 occurrences)
   - Pattern: Regular distribution
   - Likely cause: Normal user behavior
   - Impact: Low

3. **Rate limit exceeded** (345 occurrences)
   - Source IPs: 192.168.1.100, 10.0.0.50
   - Pattern: Burst traffic
   - Impact: Medium

Timeline Analysis

## Error Timeline

08:00 - Normal operations (5-10 errors/min)
09:00 - Database connection errors spike (200+ errors/min)
09:15 - Peak error rate (234 errors/min)
09:30 - Database connection restored
10:00 - Return to normal (8-12 errors/min)

## Correlation
- Traffic increased 300% at 09:00
- Database CPU at 95% during incident
- Connection pool exhausted

Performance Metrics

## Response Times (from logs)

**Average**: 234ms
**P50**: 180ms
**P95**: 450ms
**P99**: 890ms

**Slow Requests** (>1s):
- /api/search: 2.3s avg (45 requests)
- /api/reports: 1.8s avg (23 requests)

**Fast Requests** ( 10/min
2. **Response Time P95** > 500ms
3. **Error Rate** > 2%
4. **Memory Usage** > 80%
5. **Rate Limit Hits** > 100/hour from single IP

Analysis Techniques

Regular Expression Patterns

# Find all errors
grep -E "ERROR|Exception|Failed" app.log

# Extract timestamps and errors
grep "ERROR" app.log | awk '{print $1, $2, $4}'

# Count error types
grep "ERROR" app.log | cut -d':' -f2 | sort | uniq -c | sort -nr

# Find slow requests
awk '$7 > 1000 {print $0}' access.log  # Response time > 1s

Time-Based Analysis

# Errors per hour
awk '{print $1" "$2}' app.log | cut -d':' -f1 | uniq -c

# Peak error times
grep "ERROR" app.log | cut -d' ' -f2 | cut -d':' -f1 | sort | uniq -c | sort -nr

Tools Integration

• Elasticsearch + Kibana: Centralized logging and visualization
• Splunk: Enterprise log management
• Datadog: APM and log analysis
• CloudWatch: AWS log aggregation
• Grafana Loki: Open-source log aggregation
• Papertrail: Simple log management

Notes

• Always consider log volume and retention
• Implement log rotation and archiving
• Use structured logging (JSON) for easier parsing
• Include request IDs for distributed tracing
• Set up alerts for critical error patterns
• Regular log analysis prevents incidents
• Correlation with metrics provides better insights

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: CuriousLearner
• Source: CuriousLearner/devkit
• License: MIT

Install and usage instructions live in the source repository linked above.