Coverage Analyzer

Coverage Analyzer Skill

Purpose

This skill provides comprehensive test coverage analysis, gap identification, threshold validation, and detailed reporting to ensure code quality and adequate test coverage across the codebase.

When to Use

• Measuring test coverage for a module or project
• Identifying untested code and coverage gaps
• Validating coverage meets threshold (≥ 80%)
• Generating coverage reports for review
• Analyzing coverage trends over time
• Finding missing branch coverage

Coverage Analysis Workflow

1. Basic Coverage Measurement

Run Tests with Coverage:

# Install pytest-cov if not installed
pip install pytest-cov

# Basic coverage measurement
pytest --cov=src tests/

# Coverage with terminal report
pytest --cov=src --cov-report=term tests/

# Coverage with missing lines
pytest --cov=src --cov-report=term-missing tests/

# Coverage for specific module
pytest --cov=src.tools.feature tests/

Quick Coverage Check:

# Show percentage only
pytest --cov=src --cov-report=term tests/ | grep TOTAL

# Check if coverage meets threshold
pytest --cov=src --cov-fail-under=80 tests/

# Quiet mode with coverage
pytest --cov=src --cov-report=term -q tests/

Deliverable: Basic coverage metrics

---

2. Detailed Coverage Analysis

Generate Detailed Reports:

# HTML coverage report (most detailed)
pytest --cov=src --cov-report=html tests/
# Open htmlcov/index.html in browser

# XML coverage report (for CI/CD)
pytest --cov=src --cov-report=xml tests/

# JSON coverage report
pytest --cov=src --cov-report=json tests/

# Multiple report formats
pytest --cov=src \
  --cov-report=html \
  --cov-report=xml \
  --cov-report=term-missing \
  tests/

Analyze Coverage by Module:

# Show coverage for each file
pytest --cov=src --cov-report=term tests/

# Coverage with line numbers
pytest --cov=src --cov-report=term-missing tests/

# Annotated source files
pytest --cov=src --cov-report=annotate tests/
# Creates .py,cover files with coverage annotations

Deliverable: Detailed coverage reports

---

3. Coverage Gap Identification

Find Untested Code:

# Show missing lines
pytest --cov=src --cov-report=term-missing tests/

# Show missing branches
pytest --cov=src --cov-branch --cov-report=term-missing tests/

# Generate HTML report to visualize gaps
pytest --cov=src --cov-report=html tests/
# Open htmlcov/index.html to see line-by-line coverage

Identify Low Coverage Modules:

# Run coverage and parse output
pytest --cov=src --cov-report=term tests/ | grep -v "100%"

# Find files below 80% coverage
pytest --cov=src --cov-report=term tests/ | awk '$NF ~ /%/ && $NF+0 "

Analyze Conditional Coverage:

# Find untested conditionals
pytest --cov=src --cov-branch --cov-report=term-missing tests/ | grep "if"

# Check switch/case coverage
pytest --cov=src --cov-branch --cov-report=term-missing tests/ | grep "case"

Deliverable: Branch coverage metrics

---

5. Coverage Threshold Validation

Validate Minimum Coverage:

# Fail if coverage below 80%
pytest --cov=src --cov-fail-under=80 tests/

# Fail if coverage below 90%
pytest --cov=src --cov-fail-under=90 tests/

# Check with specific threshold and report
pytest --cov=src --cov-fail-under=80 --cov-report=term-missing tests/

Per-Module Thresholds:

# Core logic: 90% minimum
pytest --cov=src.core --cov-fail-under=90 tests/

# Utilities: 85% minimum
pytest --cov=src.utils --cov-fail-under=85 tests/

# Interfaces: 80% minimum
pytest --cov=src.interfaces --cov-fail-under=80 tests/

Deliverable: Threshold validation report

---

6. Coverage Configuration

Configure .coveragerc:

# .coveragerc
[run]
source = src
omit =
    tests/*
    */__init__.py
    */venv/*
    */.venv/*
    */migrations/*

branch = True

[report]
exclude_lines =
    # Default pragmas
    pragma: no cover

    # Don't complain about missing debug-only code
    def __repr__
    def __str__

    # Don't complain if tests don't hit defensive assertion code
    raise AssertionError
    raise NotImplementedError

    # Don't complain if non-runnable code isn't run
    if __name__ == .__main__.:
    if TYPE_CHECKING:

    # Don't complain about abstract methods
    @abstractmethod

precision = 2
show_missing = True

[html]
directory = htmlcov

[xml]
output = coverage.xml

Configure pyproject.toml:

# pyproject.toml
[tool.coverage.run]
source = ["src"]
omit = [
    "tests/*",
    "*/__init__.py",
    "*/venv/*",
    "*/.venv/*",
]
branch = true

[tool.coverage.report]
exclude_lines = [
    "pragma: no cover",
    "def __repr__",
    "raise AssertionError",
    "raise NotImplementedError",
    "if __name__ == .__main__.:",
    "if TYPE_CHECKING:",
    "@abstractmethod",
]
precision = 2
show_missing = true
fail_under = 80

[tool.coverage.html]
directory = "htmlcov"

[tool.coverage.xml]
output = "coverage.xml"

Deliverable: Coverage configuration

---

7. Coverage Reporting

Generate Coverage Reports:

# Terminal report
pytest --cov=src --cov-report=term tests/

# Terminal report with missing lines
pytest --cov=src --cov-report=term-missing tests/

# HTML report for detailed analysis
pytest --cov=src --cov-report=html tests/

# XML report for CI/CD integration
pytest --cov=src --cov-report=xml tests/

# JSON report for programmatic access
pytest --cov=src --cov-report=json tests/

# All reports
pytest --cov=src \
  --cov-report=term-missing \
  --cov-report=html \
  --cov-report=xml \
  --cov-report=json \
  tests/

Coverage Summary Report:

# Coverage Report

## Overall Coverage: 87%

### By Module
| Module | Coverage | Missing Lines |
|--------|----------|---------------|
| src.core | 95% | 45-47, 89 |
| src.utils | 88% | 12-15, 67-70 |
| src.interfaces | 82% | 23, 56-58 |
| src.models | 100% | - |

### Critical Gaps
1. Error handling in `src.core.process()` (lines 45-47)
2. Edge case validation in `src.utils.validate()` (lines 67-70)
3. Exception paths in `src.interfaces.execute()` (lines 56-58)

### Recommendations
- Add tests for error conditions in core module
- Improve edge case coverage in utilities
- Test exception handling in interfaces

### Status
✅ Overall coverage: 87% (target: 80%)
✅ Core business logic: 95% (target: 90%)
✅ Utilities: 88% (target: 85%)
✅ All thresholds met

Deliverable: Comprehensive coverage reports

---

8. Coverage Trend Analysis

Track Coverage Over Time:

# Generate coverage badge
pip install coverage-badge
coverage-badge -o coverage.svg

# Save coverage history
pytest --cov=src --cov-report=json tests/
mv coverage.json coverage-$(date +%Y%m%d).json

# Compare coverage between runs
coverage json
python -c "import json; print(json.load(open('coverage.json'))['totals']['percent_covered'])"

Coverage Diff:

# Coverage for current branch
pytest --cov=src --cov-report=json tests/
cp coverage.json coverage-current.json

# Coverage for main branch
git checkout main
pytest --cov=src --cov-report=json tests/
cp coverage.json coverage-main.json

# Compare
diff 90% coverage on new code
- Review coverage reports locally
- Fix coverage gaps before committing

### Pre-Commit
- Verify coverage meets threshold
- Review coverage diff vs main branch
- Ensure new code is well tested
- Check branch coverage for complex logic

### CI/CD
- Generate coverage reports for every build
- Fail build if coverage below threshold
- Track coverage trends over time
- Archive coverage reports

### Review
- Focus on critical paths first
- Prioritize business logic coverage
- Don't chase 100% coverage unnecessarily
- Exclude unreachable code appropriately

---

## Coverage Gap Prioritization

### High Priority (Fix Immediately)
- Core business logic uncovered
- Error handling paths untested
- Security-critical code uncovered
- Data validation missing tests
- API endpoints without tests

### Medium Priority (Fix Soon)
- Utilities with <80% coverage
- Branch coverage gaps in logic
- Edge cases not tested
- Configuration parsing untested

### Low Priority (Fix When Possible)
- Representation methods (__repr__, __str__)
- Debug/logging code
- CLI help text
- Non-critical utilities

### Acceptable to Exclude
- Abstract methods
- TYPE_CHECKING blocks
- if __name__ == "__main__"
- Platform-specific code not running in CI
- Deprecated code scheduled for removal

---

## Supporting Resources

- **pytest-cov documentation**: https://pytest-cov.readthedocs.io
- **coverage.py documentation**: https://coverage.readthedocs.io
- **Coverage best practices**: https://testing.googleblog.com
- **Sample .coveragerc**: In project root

---

## Success Metrics

- [ ] Coverage calculated accurately
- [ ] Overall coverage ≥ 80%
- [ ] Critical code coverage ≥ 90%
- [ ] Coverage gaps identified
- [ ] Reports generated successfully
- [ ] Thresholds validated
- [ ] Recommendations provided

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [matteocervelli](https://github.com/matteocervelli)
- **Source:** [matteocervelli/llms](https://github.com/matteocervelli/llms)
- **License:** MIT

Install and usage instructions live in the source repository linked above.