Llm Aiops
Use when working on AIOps (AI for IT Operations) tasks — incident management, root cause analysis, log parsing/anomaly detection, cloud infrastructure management, or applying LLMs to IT operations. Also use when researching LLM-based solutions for incident response, fault localization, or infrastructure-as-code.
Install
$ agentstack add skill-renky1025-agent-skills-llm-aiops Open-source listing — not yet scanned by AgentStack. Follow the source repository for install instructions.
Security review
⚠ Flagged1 finding(s); flagged for manual review. · v0.1.0 How review works →
- • Prompt-injection patterns
- • Secret / credential exfiltration
- • Dangerous shell & filesystem operations
- • Untrusted network calls
- • Known-malicious package signatures
- high Dangerous shell/eval execution.
What it can access
- ✓ Network access No
- ✓ Filesystem access No
- ✓ Shell / process execution No
- ✓ Environment & secrets No
- ● Dynamic code execution Used
From automated source analysis of v0.1.0. “Used” means the capability is present in the source — more access means more to trust, not that it’s unsafe.
About
LLM AIOps Reference
Overview
Curated reference applying Large Language Models to AIOps, based on the awesome-LLM-AIOps research taxonomy. Covers incident management, log analysis, and infrastructure management.
When to Use
- Incidents: triaging, diagnosing, mitigating cloud incidents
- RCA: root cause analysis of system failures
- Logs: parsing, anomaly detection, logging statement generation
- Infrastructure: IaC, performance diagnosis, LLM training platforms
- AIOps QA: domain-specific question answering for operations
Taxonomy & Key Approaches
1. LLM for Incident Management
| Area | Key Approaches | Notable Systems | |------|---------------|-----------------| | Surveys & Benchmarks | Evaluation frameworks for AIOps agents | AIOpsLab (MLSys 2025), ITBench (ICML 2025) | | Incident Diagnosis | LLM agents + diagnostic tools | D-Bot (VLDB 2024), NetAssistant (NSDI 2024), FLASH (Preprint 2024) | | Incident Reporting | Fine-tuning, CoT prompting for summarization | Oasis (FSE 2023), COLA (ICSE-SEIP 2024), MonitorAssistant (FSE 2024) | | Root Cause Analysis | Agents with tool augmentation, CoT, ICL, multi-agent | RCAgent (CIKM 2024), mABC (EMNLP 2024), Flow-of-Action (WWW 2025), OpenRCA (ICLR 2025), COCA (ICSE 2025), RCACopilot (EuroSys 2024) | | Incident Mitigation | Agent-based with troubleshooting guides, RAG | Nissist (ECAI 2024), LLexus (SIGOPS 2024), STRATUS (NeurIPS 2025) | | Postmortem Analysis | Fine-tuning for fault profiling | FaultProfIT (ICSE-SEIP 2024), FAIL (ASE 2024) | | AIOps Q&A | Instruction tuning, RAG, domain-specialized LLMs | OWL (ICLR 2024), iKnow (ASE 2025), OpsEval (Preprint 2023), MSQA (EMNLP 2023) |
2. LLM for Log Analysis
| Area | Key Approaches | Notable Systems | |------|---------------|-----------------| | Log Parsing | ICL prompting, adaptive caching, entropy + CoT merging | DivLog (ICSE 2024), LILAC (FSE 2024), LogBatcher (ASE 2024), LUNAR (FSE 2025), LibreLog (ICSE 2025) | | Log Anomaly Detection | Prompting, RAG, fine-tuning | LogGPT (Preprint 2023), LogPrompt (ICPC 2024), RAGLog (Preprint 2024), CodeAD (2025) | | Logging Statement Generation | ICL, fine-tuning, static context analysis | UniLog (ICSE 2024), SCLogger (FSE 2024), FastLog (ISSTA 2024) |
3. LLM for Infrastructure Management
| Area | Key Approaches | Notable Systems | |------|---------------|-----------------| | Infrastructure-as-Code | Agent-based bug discovery, semantic checking | Unearthing IaC Checks (SOSP 2024) | | LLM Training Platform | Black-box performance diagnosis | LLMPrism (DSN 2025) | | Benchmarks | IaC code generation | IaC-Eval (NeurIPS D&B 2024) |
Common LLM Techniques Used
- Prompting: ICL (In-Context Learning), CoT (Chain-of-Thought), ToT (Tree-of-Thought)
- Fine-tuning: Instruction tuning, domain-specific adaptation (e.g., OWL for IT ops)
- Agent-based: Tool-augmented LLMs, multi-agent collaboration, SOP-guided agents
- RAG: Retrieval-Augmented Generation for incident resolution recommendations
- Code Synthesis: LLM generates rule code for anomaly detection (CodeAD)
Key Benchmarks
- ITBench (ICML 2025) — Diverse real-world IT automation tasks
- AIOpsLab (MLSys 2025) — Holistic autonomous cloud agent evaluation
- OpsEval — Comprehensive AIOps task benchmark
- OpenRCA (ICLR 2025) — RCA benchmark with fault propagation awareness
- IaC-Eval (NeurIPS 2024) — Infrastructure-as-Code generation benchmark
Common Mistakes
- ❌ Treating log parsing as a solved problem — LLM-based parsers still struggle with unseen log formats; use adaptive caching (LILAC pattern)
- ❌ Applying generic LLMs to ops without domain tuning — Domain-specific models (OWL) or RAG significantly outperform general-purpose prompting for AIOps Q&A
- ❌ Single-agent RCA for complex microservice failures — Multi-agent systems (mABC, Flow-of-Action) with SOP guidance outperform single-agent approaches
- ❌ Ignoring confidence estimation in RCA — Use PACE-LM style confidence calibration before acting on LLM-generated RCA results
Real-World Impact
The field has rapidly evolved from basic prompting (2023) to sophisticated multi-agent systems with tool augmentation (2024-2025), with production-grade systems like Nissist, STRATUS, and iKnow deployed in real cloud environments. Key trend: moving from single LLM calls to autonomous agent loops with verification.
Source & license
This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.
- Author: renky1025
- Source: renky1025/agent-skills
- License: MIT
Install and usage instructions live in the source repository linked above.
Reviews
No reviews yet — be the first.
Write a review
Versions
- v0.1.0 Imported from the upstream source.