Prometheus & Grafana Stack

The default OSS observability stack in 2026: Prometheus for metrics, Grafana for dashboards, Loki for logs, Tempo for traces. All free, all CNCF / Grafana Labs OSS.

Prometheus — metrics

• ★ Prometheus (CNCF) — pull-based scraper; PromQL query language; alertmanager. The default.
• Prometheus Operator — k8s CRDs for Prometheus / Alertmanager / ServiceMonitor.
• kube-prometheus-stack — the all-in-one Helm chart that everyone deploys.

Storage scaling (single Prometheus is limiting)

• ★ VictoriaMetrics — drop-in Prometheus-compatible TSDB with much better resource usage. The default for "Prometheus is OOMing."
• Thanos — global view + long-term storage on object stores; multi-cluster.
• Cortex — same idea; multi-tenant; horizontally scalable.
• Mimir (Grafana Labs) — Cortex's open-source successor; the recommended Grafana-stack choice.
• M3DB — Uber's TSDB.

Push gateway / non-scrape sources

• Prometheus Pushgateway — for batch jobs that finish before scrape.
• OpenTelemetry Collector — convert OTLP metrics to Prometheus format.
• Telegraf / Vector — agent-based metric collection.

Visualization

• ★ Grafana — universal dashboards; supports every TSDB and log store; alerting built in.
• Grafana Cloud — generous free tier; managed Mimir + Loki + Tempo + Pyroscope.
• Perses — newer dashboard tool; CNCF.

Alerting

• Alertmanager — bundled with Prometheus; routes / groups / silences alerts.
• Grafana OnCall — paging integrated with Grafana; OSS + cloud.
• Karma — alerting dashboard on top of Alertmanager.
• PagerDuty / Opsgenie — see Status & Incidents.

Exporters (the metric sources)

• ★ node_exporter — host CPU / disk / network.
• cadvisor — per-container metrics on a node.
• kube-state-metrics — k8s object state.
• postgres_exporter / mysqld_exporter / redis_exporter — DB metrics.
• blackbox_exporter — synthetic probes (HTTP / TCP / ICMP).
• smokeping_exporter — latency over time.
• Hundreds more on prometheus.io/docs/instrumenting/exporters.

OpenTelemetry alternative path

• ★ OpenTelemetry Collector — vendor-neutral; collect logs / metrics / traces; export to Prometheus / Loki / Tempo / paid backends.
• OTLP metrics — newer; many SDKs prefer it; export to Prometheus via OTel Collector.
• OTel Operator — k8s sidecar / agent injection.

Patterns to adopt

• ★ Default new k8s observability: kube-prometheus-stack + Loki + Tempo via loki-stack chart, all federated to Grafana.
• Long retention — single Prometheus dies at scale. Move to VictoriaMetrics or Mimir early.
• Don't /metrics on every endpoint with high cardinality — userId labels = exploding cardinality = OOM.
• RED metrics + USE method — Rate / Errors / Duration for services; Utilization / Saturation / Errors for resources.
• SLOs — track user-impacting metrics; set burn-rate alerts.
• Alertmanager grouping — silence storms; group by service / cluster.

Pick this if…

• Default OSS stack: Prometheus + Grafana + Loki + Tempo (all OSS, all free).
• Prometheus too small: VictoriaMetrics (drop-in) or Mimir (Grafana-stack).
• Hosted free tier: Grafana Cloud.
• Vendor-neutral instrumentation: OpenTelemetry + OTel Collector.
• Already paying Datadog: the boring vendor path; this stack is what you'd run if you stopped paying.