Detection Tuning

Detection Tuning Skill

Analyze and tune CrowdStrike NGSIEM detection rules for actionable security alerting with minimal false positives.

Purpose

Transform raw out-of-the-box (OOTB) detection templates into production-ready rules by:

1. Applying environmental context (user population, infrastructure, baseline patterns)
2. Integrating available CQL enrichment functions for identity classification
3. Recommending threshold and exclusion tuning based on false positive patterns
4. Generating analyst-ready YAML templates

Analysis Workflow

Step 1: Read the Detection Template

# Read the target detection
cat resources/detections//.yaml

Extract and understand:

• Vendor/Data Source: AWS CloudTrail, EntraID, SASE, Google, CrowdStrike, GitHub
• Detection Logic: What events trigger alerts
• Current Thresholds: Count thresholds, time windows
• Existing Exclusions: Any commented or active filters

Step 2: Identify Tuning Opportunities

Reference [ENVIRONMENTCONTEXT.md](ENVIRONMENTCONTEXT.md) to understand:

• User Population: ~500 users, primarily US-based across all timezones
• High-Risk Users: Executives and engineers (Mac users with elevated access)
• Infrastructure: 100% cloud (11 AWS accounts, EntraID, Google Workspace, GitHub)
• Normal Patterns: Business hours activity, SASE VPN connections, SSO logins
• GitHub Activity: Service account patterns (merge-queue, dependabot, Actions automation)
• Statistical Baselines: For 500-user environment, consider 30-60 day baselines for establishing normal behavior
• Privilege Context: TEAM users (PAM system), global admins, engineering groups with elevated access

Step 2.5: Pre-Activation Historical Query (when activating an inactive or new detection)

Run this step before setting status: active on any detection. Skip only for detections targeting rare/clearly malicious TTPs (credential dumping, crypto miners) where expected volume is near-zero, or for log sources with fewer than 7 days of history.

Why: Detections can look correct in code review but still be noisy against real data. Merge exclusion bugs, missing service account filters, and overly broad regex are only visible through historical queries. A detection disabled for noise often has no documented reason — the gut feeling that something was noisy is the only signal.

Process

1. Run the filter as a 30d historical query via ngsiem_query with start_time="30d"
2. Classify the results — group by actor, operation type, and key event/commit pattern:

`` | groupBy([actor_field, operation_field, pattern_field], function=[ count(as=Count), collect([message_or_event_field], limit=10) ]) | sort(Count, order=desc) ``

1. Identify FP patterns — what proportion is expected workflow vs. genuine anomaly? Common patterns:

• High-volume actors that are automation/service accounts
• Commit/event message patterns indicating normal operations (PR merges, sync commits, scheduled jobs)
• Known business workflows (CI/CD deployments, admin provisioning, release processes)

1. Propose exclusions for FP patterns before activating — present diffs for approval
2. Confirm acceptable volume after exclusions applied

Volume Guidance (hits/30d after exclusions)

| Count | Action | |-------|--------| | 0–15 | Activate | | 15–50 | Review patterns — add exclusions if FP-heavy | | 50+ | Do not activate — filter logic needs narrowing first |

Target: 0–5 alerts/day environment-wide. A single noisy detection burns analyst time and erodes confidence in all alerts.

Document the Baseline

After completing pre-tuning, add a brief comment to the detection's description field or TUNING_BACKLOG.md:

# Pre-tuning baseline (YYYY-MM-DD): ~N genuine events/30d after exclusions

Step 3: Apply Enrichment Functions

Reference [AVAILABLEFUNCTIONS.md](AVAILABLEFUNCTIONS.md) to add context. We have 38 available functions across multiple vendors:

Universal Identity Enrichment

For cross-platform identity enrichment:

// Enrich AWS events with EntraID identity data
#repo=cloudtrail
| $aws_enrich_user_identity()
| UserEmail := lower(UserIdentity)
| $identity_enrich_from_email()
| IsAdmin="True"
| HasProdAccess="True"

// Enrich generic vendor events
| UserEmail := lower(user.email)
| $identity_enrich_from_email()
| Department=*

Available function:

• $identity_enrich_from_email() - Cross-platform identity enrichment (requires UserEmail field)

AWS Detections

For AWS CloudTrail detections:

// Core identity enrichment
| $aws_enrich_user_identity()
| $aws_classify_identity_type(include_service_detection="true")
| IsHumanIdentity=true  // Focus on human actors

// Service account filtering (8 service account types)
| $aws_service_account_detector()
| ServiceAccountType!="CodeBuild"  // Customize per detection

// Cross-account trust validation
| $aws_validate_cross_account_trust()
| $aws_classify_account_trust()
| TrustClassification="EXTERNAL"

// Service provider IP detection
| $aws_trusted_ip_detector()
| IsTrustedServiceIP=false

// Session context extraction
| $aws_extract_session_context()
| SessionName=*

Available functions (7):

• $aws_enrich_user_identity() - Extract CloudTrail user identity
• $aws_classify_identity_type() - Human vs service classification
• $aws_service_account_detector() - Detect 8 service account types (CodeBuild, Lambda, etc.)
• $aws_validate_cross_account_trust() - Trust relationship validation
• $aws_classify_account_trust() - Account trust classification (INTERNAL/EXTERNAL/UNKNOWN)
• $aws_trusted_ip_detector() - Service provider IP detection
• $aws_extract_session_context() - Session metadata extraction

GitHub Detections

For GitHub push events and repository activity:

// Core enrichment - CALL THIS FIRST
| $github_enrich_event_context()

// Service account filtering (per-detection customization)
| $github_service_account_detector()
| ServiceAccountType!="merge-queue"  // Customize based on detection needs
| ServiceAccountType!="dependabot"

// Or use all-or-nothing exclusion filtering
| $github_apply_exclusions()
| IsExcluded=false

// Risk detection (depends on github_enrich_event_context)
| $github_flag_risky_operations()
| IsRiskyOperation=true

Available functions (5):

• $github_enrich_event_context() - Core push event enrichment (CALL FIRST - required by other functions)
• $github_classify_sender_type() - Human vs bot classification
• $github_service_account_detector() - Per-detection service account filtering (merge-queue, dependabot, actions-bot, etc.)
• $github_flag_risky_operations() - Risk scoring (depends on githubenrichevent_context)
• $github_apply_exclusions() - All-or-nothing bot filtering

EntraID Detections

For EntraID signin and audit events:

Basic Identity Enrichment:

// Core identity extraction
| $entraid_enrich_user_identity()
| $entraid_classify_user_type()  // v2.0 with enhanced classification
| UserType="Employee"

// HR data enrichment
| $entraid_lookup_user_mapping()
| Department=*
| IsActive="True"

Group & Privilege Analysis:

// Comprehensive group membership
| $entraid_enrich_group_summary()
| TotalGroups > 0

// Check privileged groups (tier-based)
| $entraid_check_privileged_groups(strict_mode="true")
| IsPrivilegedUser=true
| PrivilegeTier=*

// Check TEAM (PAM) eligibility
| $entraid_check_team_eligibility()
| TEAMViolation=true

// Validate department access patterns
| $entraid_validate_department_access(validate_technical="true")
| DepartmentAccessViolation=true

Authorization Context:

// Trust level enrichment
| $entraid_lookup_trust_level()
| TrustLevel=*

// Authorization context
| $entraid_add_authorization_context()
| AuthorizationContext=*

// Policy violation detection
| $entraid_flag_unauthorized_actions()
| IsUnauthorized=true

// Admin enforcement filter
| $entraid_require_admin_authorization()
| RequiresAdminAuth=true

Investigative Functions (Parameterized):

// Full signin audit for specific user
| $entraid_user_signin_audit(user="user@example.com")

// Device inventory for user
| $entraid_user_device_summary(user="user@example.com")

// Mobile signin history
| $entraid_user_mobile_signins(user="user@example.com")

// Unregistered device detection
| $entraid_user_unregistered_devices(user="user@example.com")

Available functions (15):

Basic Identity:

• $entraid_enrich_user_identity() - Extract user identity
• $entraid_classify_user_type() - v2.0 enhanced classification (service/contractor/employee)
• $entraid_lookup_user_mapping() - HR data enrichment

Group & Privilege:

• $entraid_enrich_group_summary() - Comprehensive group analysis
• $entraid_check_privileged_groups() - Privilege tier checking (strict_mode parameter)
• $entraid_check_team_eligibility() - TEAM/PIM eligibility tracking
• $entraid_validate_department_access() - Department hierarchy validation

Authorization:

• $entraid_lookup_trust_level() - Trust level enrichment
• $entraid_add_authorization_context() - Auth context enrichment
• $entraid_flag_unauthorized_actions() - Policy violation detection
• $entraid_require_admin_authorization() - Admin enforcement filter

Investigative (parameterized):

• $entraid_user_signin_audit(user) - Full signin audit trail
• $entraid_user_device_summary(user) - Device inventory
• $entraid_user_mobile_signins(user) - Mobile signin history
• $entraid_user_unregistered_devices(user) - Unregistered device detection

Network-Based Detections

For SASE SASE and network traffic:

// Trusted network detection (SASE VPN)
| $trusted_network_detector(extend_trust="true", include_private="true")
| IsExcluded=false  // Filter out SASE VPN traffic

// SASE + EntraID enrichment (30+ fields)
| $sase_enrich_user_identity()
| UserEmail=*

// Connection source validation
| $sase_validate_connection_source()
| IsValidConnectionSource=true

// Geographic risk scoring
| $score_geo_risk()
| FinalShouldAlert=true

Available functions (4):

• $trusted_network_detector() - SASE VPN filtering (extendtrust, includeprivate parameters)
• $sase_enrich_user_identity() - SASE + EntraID enrichment (30+ fields)
• $sase_validate_connection_source() - Connection source validation
• $score_geo_risk() - Geographic risk scoring

Statistical Baseline Detection

For establishing normal behavior baselines:

// Establish 30-day baseline
| defineTable("baseline_stats", [
    groupBy([EntityId, EventType], function=[
        avg(HourlyCount, as=BaselineAvg),
        stddev(HourlyCount, as=BaselineStdDev)
    ])
], lookbackDays=30, excludeStart=2h)

// Calculate dynamic threshold
| match(file="baseline_stats", field=[EntityId, EventType])
| Threshold := BaselineAvg + 3 * BaselineStdDev
| test(CurrentCount > Threshold)

// Or use pre-built baseline functions
| $create_baseline_60d()  // 60-day lookback

Available functions (3):

• $create_baseline_7d() - 7-day historical baseline
• $create_baseline_60d() - 60-day historical baseline
• $create_baseline_90d() - 90-day historical baseline

Step 4: Generate Tuned Output

Produce three deliverables:

1. Analysis Report: Document findings and rationale
2. Tuning Recommendations: Specific CQL snippets with explanations
3. Production-Ready YAML: Complete template ready for deployment

Standard Detection Format

name: "Detection Name - Tuned"
resource_id: detection_resource_id
description: |
  [Enhanced description with tuning notes]

  Tuning Applied:
  - [List of tuning changes]

severity: [Adjusted severity 5-90]
status: active
mitre_attack: ["TA00XX:T1XXX"]  # Format: ["Tactic (TAXXXX):Technique: Sub-technique (T1XXX.YYY)"]
search:
  filter: |
    [Tuned query with enrichment functions]
  lookback: [Adjusted time window]
  trigger_mode: summary
  outcome: detection
operation:
  schedule:
    definition: '@every [frequency]'

Multi-Tier Severity Detection Format

For detections with different severity levels based on threshold/context:

name: "Detection Name - Multiple Thresholds"
resource_id: detection_multi_tier
severity: 50  # Default STANDARD tier
description: |
  Detection with tiered thresholds for different attack patterns:
  - RAPID: High-confidence attacks (Severity 70) - 3+ events in 15 minutes
  - STANDARD: Balanced detection (Severity 50) - 5+ events in 30 minutes
  - SUSTAINED: Slow attacks (Severity 40) - 8+ events in 60 minutes

  Tuning Applied:
  - Multi-tier severity based on velocity
  - Service account exclusions
  - Geographic risk scoring

severity: 50
status: active
mitre_attack: ["TA0001:T1078"]
search:
  filter: |
    #repo=cloudtrail event.name="ConsoleLogin"
    | $aws_enrich_user_identity()
    | $aws_classify_identity_type(include_service_detection="true")
    | IsHumanIdentity=true

    // Count events per user
    | groupBy([UserIdentity], function=[count(as=Count)])

    // Calculate time window
    | DurationMinutes := duration(start=_earliest, end=_latest, unit="minutes")

    // Multi-tier severity
    | case {
        test(Count >= 3) test(DurationMinutes = 5) test(DurationMinutes = 8) test(DurationMinutes  0
  lookback: 2h
  trigger_mode: summary
  outcome: detection
operation:
  schedule:
    definition: '@every 15m'

ADS Metadata Updates During Tuning

When the detection template has an ads: block, update the following fields as part of the tuning change:

1. Append new FP patterns to ads.false_positives:

```yaml false_positives: # ... existing entries ...

• pattern: ""

characteristics: "" tuning: "" status: "tuned" ```

1. Update ads.strategy_abstract if enrichment functions were added or the detection logic changed meaningfully. Append a note about the new enrichment, don't rewrite from scratch.

1. Update ads.blind_spots if the tuning introduces new limitations (e.g., "Excludes all activity from service account X" means attacks using that account are now blind).

1. Set date and author fields:

``yaml ads_updated: "YYYY-MM-DD" # Today's date ads_author: "detection-tuning" ``

1. Append a structured entry to knowledge/tuning/tuning-log.md after every tuning action:

```markdown ## YYYY-MM-DD —

Trigger: Change: Before: ` **After:** **Alerts:** [] **Validation:** **PR:** # ``

If the detection template does NOT have an ads: block, do not add one during tuning — ADS backfill happens during triage (Phase 4 of the SOC skill).

Step 5: Behavioral Rule Tuning (for correlate() rules)

For behavioral rules using correlate(), additional tuning considerations apply:

Time Window Optimization (within):

correlate(
  EventA: { ... },
  EventB: { ... | field  EventA.field },
  within=30m  // Tune based on expected attack duration
)

• Too narrow: May miss legitimate attack chains
• Too wide: Increases false positives from unrelated events

Sequence Enforcement (sequence):

• Use sequence=true only when event order is attack-relevant
• Non-sequence mode is more flexible for correlation

Output Outcome Types: | Outcome | Field Value | Use Case | |---------|-------------|----------| | Behavioral Detection | Ngsiem.event.outcome="behavioral-detection" | Multi-event attack patterns | | Correlation Rule Detection | Ngsiem.event.outcome="correlation-rule-detection" | Single-event threshold rules | | Behavioral Case | Ngsiem.event.outcome="behavioral-case" | Case-generating rules |

Behavioral Rule YAML Template:

name: "Behavioral Detection - Attack Chain"
resource_id: behavioral_attack_chain
severity: 70
search:
  filter: |
    correlate(
      Step1: { ... },
      Step2: { ..

…

## Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

- **Author:** [willwebster5](https://github.com/willwebster5)
- **Source:** [willwebster5/agent-skills](https://github.com/willwebster5/agent-skills)
- **License:** MIT

Install and usage instructions live in the source repository linked above.