Otel Collector

OpenTelemetry Collector

This skill covers the configuration surface of individual OpenTelemetry Collector components. It targets opentelemetry-collector and opentelemetry-collector-contrib.

It does not cover OTTL expressions (see otel-ottl), declarative SDK configuration (otel-declarative-config), or end-to-end pipeline design choices. Reach for those skills when the question is about transformation language, SDK setup, or pipeline composition.

Workflow

1. Identify the component. Find the type in the user's config or question (log_dedup, interval, otlp, …). Note that several components were renamed to snake_case in v0.150.0–v0.151.0 with deprecated aliases preserved — see [Recent renames](#recent-renames).
2. Load the component page. If the component is in the [Component index](#component-index), read components//README.md first — it carries the metadata, description, main use-cases, and a Details index. Open the linked detail files (configuration.md, verification.md, advanced.md, quirks.md, …) only as the question requires; don't load files you don't need.
3. If the component is not indexed, say so explicitly and fall back to the upstream README under processor//, receiver//, exporter//, connector//, or extension// in opentelemetry-collector-contrib. Don't invent config keys from memory — Collector components evolve quickly.
4. Apply Collector-wide conventions. Named instances (type/name), stability levels, and pipeline placement rules in [Collector-wide conventions](#collector-wide-conventions) apply to every component.
5. Verify. Run the component page's Verification recipe — telemetrygen (see the otel-telemetrygen skill) plus a debug or file exporter — to confirm the component behaves as the docs claim. Alpha- and Development-stability components are common here, and behavior changes between releases. See [Verification harness](#verification-harness) for how to run a recipe end-to-end.

Component index

Each component is a directory under components//. The File column points at the lean README.md (metadata, description, main use-cases, and a Details index); the full config reference, verification recipe, advanced use-cases, and quirks live in on-demand detail files linked from that README.

| Type | File | Kind | Signals | Stability | Summary | |------|------|------|---------|-----------|---------| | log_dedup | components/log_dedup/README.md | processor | logs | Alpha | Deduplicates identical log records over a time window; emits one aggregated log with a count. Renamed from logdedup in v0.151.0; alias preserved. | | interval | components/interval/README.md | processor | metrics | Alpha | Buffers cumulative monotonic metrics (and optionally gauges/summaries) and emits the latest value once per interval. Delta and non-monotonic sums pass through unchanged. | | tail_sampling | components/tail_sampling/README.md | processor | traces | Beta | Buffers whole traces and makes a single keep/drop decision after a wait window via policies. Requires a loadbalancing layer to scale across instances. | | drain | components/drain/README.md | processor | logs | Alpha | Clusters log bodies with the Drain algorithm and annotates each record with a derived template string. | | redaction | components/redaction/README.md | processor | traces, logs, metrics | Beta (traces), Alpha (logs/metrics) | Allow/block-list masking or removal of sensitive attribute keys and values, with hashing and URL/DB sanitizers. | | filter | components/filter/README.md | processor | traces, metrics, logs | Alpha | Drops spans, metric data points, and log records that match OTTL conditions (or legacy metric-name / severity filters). The most direct telemetry-volume lever. | | transform | components/transform/README.md | processor | traces, metrics, logs | Beta | Applies OTTL statements to mutate spans, metric data points, and log records in place (rename/redact/convert/aggregate attributes). The OTTL language itself lives in the otel-ottl skill. | | probabilistic_sampler | components/probabilistic_sampler/README.md | processor | traces, logs | Beta (traces), Alpha (logs) | Head sampling — deterministically keeps a configured percentage of traces/logs by hashing the trace ID (or an attribute). The cheaper, stateless counterpart to tail_sampling. | | attributes | components/attributes/README.md | processor | traces, metrics, logs | Beta | Modifies span/log/datapoint attributes via an ordered action list (insert/update/upsert/delete/hash/extract/convert), scoped by include/exclude matching. | | resource | components/resource/README.md | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Development (profiles) | Modifies the resource attributes of telemetry via the same action grammar as attributes (e.g. set service.name, drop noisy resource keys). No include/exclude matching. | | k8s_attributes | components/k8s_attributes/README.md | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Development (profiles) | Enriches telemetry with Kubernetes pod/namespace/node/workload metadata, associating each record to a pod by IP or resource attribute. Renamed from k8sattributes in v0.146.0; alias preserved. | | routing | components/routing/README.md | connector | traces, metrics, logs | Alpha | Routes same-signal telemetry to different pipelines by OTTL condition/statement per context; ordered table, first match wins, default_pipelines fallback. Replaces the deprecated routingprocessor. | | memory_limiter | components/memory_limiter/README.md | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Alpha (profiles) | Safety valve against OOM: refuses data with backpressure when Go heap exceeds a soft/hard limit, forcing GC above the hard limit. Belongs first in every pipeline; pairs with GOMEMLIMIT. | | load_balancing | components/load_balancing/README.md | exporter | traces, logs, metrics | Beta (traces/logs), Development (metrics) | Distributes telemetry across downstream Collectors via a consistent-hash ring keyed on trace ID / service.name (etc.), pinning related records to one backend. The standard way to scale tail_sampling/span_metrics. Renamed from loadbalancing in v0.153.0; alias preserved. | | otlp (receiver) | components/otlp/README.md | receiver | traces, metrics, logs, profiles | Stable (traces/metrics/logs), Development (profiles) | The canonical OTLP ingress over gRPC (4317) and/or HTTP (4318, protobuf + JSON). protocols: block, at least one required. Default endpoint is localhost, not 0.0.0.0 — must be set explicitly to receive containerized traffic. | | otlp_grpc (exporter) | components/otlp_exporter/README.md | exporter | traces, metrics, logs, profiles | Stable (traces/metrics/logs), Development (profiles) | Sends OTLP over gRPC to a downstream endpoint; gzip by default. Built-in sending_queue.batch (flush 200ms / 8192 items) replaces the batch processor, plus retry_on_failure. Renamed from otlp in core v1.50.0; alias preserved. | | file_log (receiver) | components/file_log/README.md | receiver | logs | Beta | Tails log files (glob include), turns each line/entry into a log record, and parses it via a stanza operators pipeline (json/regex/severity/timestamp/recombine). Fingerprint + offset tracking across rotation; storage for durable offsets. start_at defaults to end (existing files look empty). Renamed from filelog in v0.149.0; alias preserved. | | resource_detection | components/resource_detection/README.md | processor | traces, metrics, logs, profiles | Beta (traces/metrics/logs), Development (profiles) | Auto-detects resource attributes from the Collector's environment (host, cloud metadata services, k8s, OTEL_RESOURCE_ATTRIBUTES) via an ordered detectors list; override governs detected-vs-incoming. A failed detector stops startup. Renamed from resourcedetection in v0.153.0; alias preserved. | | file_storage | components/file_storage/README.md | extension | — (not pipeline-scoped) | Beta | Persists component state to local disk (bbolt) so it survives a Collector restart: receiver read offsets (storage:), exporter persistent send queues (sending_queue.storage:), and stateful-processor decision caches. One bbolt file per consumer; directory must exist unless create_directory: true. Not placed in a pipeline — listed under service.extensions: and referenced by other components. | | prometheus (receiver) | components/prometheus/README.md | receiver | metrics | Beta | Scrapes Prometheus-format HTTP endpoints and converts them to OTLP metrics, embedding prometheus/prometheus's scrape manager — the config: block is the same YAML as prometheus.yml (global, scrape_configs, scrape_config_files, service discovery, relabeling). Extra keys: trim_metric_suffixes, target_allocator (sharded targets from the OTel Operator), api_server (debug API). Scrape-only: rejects remote_write/remote_read/rule_files/alerting. Stateful, does not auto-shard across replicas. | | prometheus (exporter) | components/prometheus_exporter/README.md | exporter | metrics | Beta | Exposes pipeline metrics in Prometheus exposition format on a /metrics HTTP endpoint for a Prometheus server to scrape (pull, not push — contrast prometheusremotewrite). Embeds confighttp.ServerConfig; endpoint required (no default). In-memory accumulator keeps the latest value per series, dropping it after metric_expiration (5m). namespace, const_labels, send_timestamps, resource_to_telemetry_conversion, enable_open_metrics (exemplars), without_scope_info; add_metric_suffixes deprecated in favor of translation_strategy. Same prometheus type string as the receiver, different class. | | prometheus_remote_write (exporter) | components/prometheus_remote_write/README.md | exporter | metrics | Beta | Pushes OTLP metrics out as Prometheus Remote Write requests to RW-compatible backends (Cortex, Mimir, Thanos) — the opposite direction of the prometheus exporter. Type renamed prometheusremotewrite→prometheus_remote_write in v0.154.0 (deprecated alias warns). Speaks RW1 by default (protobuf_message; RW2 behind a feature gate, in development). TLS on by default (tls.insecure to disable), snappy-only compression, remote_write_queue (not sending_queue). endpoint required. Drops non-cumulative monotonic sums, histograms, and summaries under RW1. namespace, external_labels, target_info, resource_to_telemetry_conversion, disable_scope_info, add_metric_suffixes/translation_strategy, wal, max_batch_size_bytes, max_batch_request_parallelism. |

Collector-wide conventions

Named instances

Every component type supports the type/name pattern so the same type can be configured more than once. The pipeline references the named form:

processors:
  log_dedup/health-checks:
    interval: 30s
    conditions:
      - 'attributes["log.type"] == "health_check"'
  log_dedup/access-logs:
    interval: 10s

service:
  pipelines:
    logs/health:
      processors: [log_dedup/health-checks]
    logs/access:
      processors: [log_dedup/access-logs]

Stability levels

Components publish a stability level per signal. Treat these as load-bearing when recommending production use:

| Level | Use in | |-------|--------| | Development | Tests and prototypes only — breaking changes expected. | | Alpha | Limited, non-critical workloads — config keys can still change. | | Beta | Production viable — breaking changes rare. | | Stable | Production — backward compatibility guaranteed. |

Stability now varies per component — and per signal for multi-signal components (e.g. redaction is Beta for traces but Alpha for logs/metrics). Don't assume a single level for the indexed set: check each component's README header metadata table for the authoritative stability and surface it when the user asks about production readiness.

Recent renames

Many components were renamed to snake_case in v0.150.0–v0.151.0. The legacy names remain as deprecated aliases — old configs keep working but new configs should use the new names. Check the upstream component README for the exact rename version before flagging a config as broken.

Examples: logdedup → log_dedup, hostmetrics → host_metrics, spanmetrics → span_metrics, servicegraph → service_graph, k8sattributes → k8s_attributes, plus several _log and _check receivers.

Pipeline placement

Two rules of thumb that apply across components:

• memory_limiter belongs first in any processor list, before anything that allocates buffers (log_dedup, transform, tail_sampling, …).
• Batching is now done by the exporter's sending_queue.batch, not by a separate batch processor. Don't add batch to new pipelines.

Verification harness

Each component page's Verification section gives a config, a telemetrygen command, and the expected output. To run any of them:

1. Save the YAML to a file and start a collector that bundles the component — for components in the contrib/k8s distributions, otelcol-contrib --config .yaml; for components not in any distribution, build a custom collector with the OpenTelemetry Collector Builder (OCB) first.
2. Send telemetry with telemetrygen (see the otel-telemetrygen skill).
3. Watch the debug exporter's stdout (or the file exporter's output) for the expected result.

The Verification configs are minimal repros: they omit memory_limiter and other production scaffolding on purpose, to isolate the component under test. Don't copy them verbatim into production.

Adding a new component to this skill

When extending coverage:

1. Create the directory components//. Files carry no frontmatter — only SKILL.md has frontmatter.
2. Write a lean README.md — always loaded, kept small:

• Header metadata table — kind, type name, signals, per-signal stability, distributions, Go module, upstream README link, and a rename note if the component was renamed.
• Description — what the component does and the mechanism, in 1–2 tight paragraphs (push detailed mechanism/reference into configuration.md).
• Main use-cases — "Use when" / "Avoid when".
• Related components — cross-links.
• Details index — a bullet list where each item links a detail file followed by an em dash and a short description of its contents (e.g. - [Configuration](configuration.md) — config keys, defaults, validation), so the reader loads only what a question needs.

1. Split the rest into on-demand detail files under the same directory:

• configuration.md — full config-reference table (key, type, default, validation) plus any mechanism/reference tables.
• verification.md — a telemetrygen + debug/file exporter recipe that proves the documented behavior; cross-reference the otel-telemetrygen skill. Verify every telemetrygen flag against that skill — never assert a flag that doesn't exist. If telemetrygen can't produce the input the component needs, say so and point to an alternative (OTTL/transform, a custom emitter). Keep the config a minimal repro (see [Verification harness](#verification-harness)).
• advanced.md — named instances, multi-pipeline setups, combinations, and edge configs.
• quirks.md — gotchas, stability caveats, memory model, a validation-error→fix table, anti-patterns, and troubleshooting.
• Split a heavy

…

Source & license

This open-source skill is cataloged on AgentStack and links to its original source — we do not rehost the code.

• Author: ollygarden
• Source: ollygarden/opentelemetry-agent-skills
• License: Apache-2.0

Install and usage instructions live in the source repository linked above.