Tooling

Container Host OSes

Talos, Bottlerocket, Flatcar, Fedora CoreOS — minimal immutable OSes for clusters.

A normal Linux distro has thousands of packages. A container host doesn't need most of them — it needs a kernel, a container runtime, and a way to be configured declaratively. Immutable OSes give you that.

The candidates

  • Talos Linux (Sidero) — API-driven, no SSH, no shell on host; configured entirely via API. The default for new k8s self-host setups. Designed for k8s.
  • Flatcar Container Linux (Microsoft / CNCF) — successor to CoreOS; immutable, A/B partition updates. Mature.
  • Bottlerocket (AWS) — minimal; designed for hosting containers; especially on EKS / ECS.
  • Fedora CoreOS — Red Hat's continuation of CoreOS lineage.
  • K3OS — was a thing; effectively replaced by Talos / Flatcar for k8s use.
  • openSUSE MicroOS — transactional, immutable; good for k8s with Rancher / RKE2.
  • HarvesterHCI — Rancher's hyperconverged k8s + KubeVirt OS.

What "immutable" means here

  • Read-only / — apps can't modify the OS. Updates ship as a whole image.
  • Atomic updates — A/B partitions; reboot to roll back.
  • Declarative config — Ignition / cloud-init / Talos's machine config — not "ssh in and edit /etc."
  • No package manager on the host. Workloads run in containers; everything else is gone.

Talos-specific

  • No SSH. All control via talosctl over an authenticated API.
  • Machine config is YAML applied to nodes; full cluster spec lives in git.
  • talosctl reset is "wipe and re-image" in one command.
  • Cluster bootstrap, upgrade, OS patching — all via talosctl.
  • Pairs naturally with Cluster API and GitOps.

Flatcar / Fedora CoreOS

  • Same lineage as the original CoreOS.
  • Ignition for first-boot config.
  • systemd is alive; you can SSH in (unlike Talos).
  • A/B partition updates with auto-rollback on failure.
  • Less opinionated about Kubernetes; runs anything in containers.

Bottlerocket

  • AWS-built; minimal; comes with the AWS-flavored bits (kubelet pre-configured for EKS, ECS, etc.).
  • Update via apiclient; no SSH (you can SSM in via Admin / Control containers).
  • Pre-built for EKS and ECS.

When immutable hosts pay off

  • Production k8s — the cluster's nodes shouldn't be pets.
  • Compliance environments — declarative, reproducible config; auditable.
  • Anywhere the answer to "is the OS configured correctly?" should be kubectl apply or git status.

When normal Linux is fine

  • Single VPS / a handful of mixed-workload boxes — Ubuntu / Debian + Ansible is plenty.
  • Homelab — Proxmox + Debian VMs is friendlier.
  • "I sometimes need to install random packages" — immutable OSes will frustrate you.

Patterns to adopt

  • Pick Talos for new self-host k8s. It removes a whole class of "node drift" problems.
  • One image-version per cluster. Don't run mixed Talos versions; keep nodes uniform.
  • Cluster recreation should work. Practice it. If it doesn't, your cluster has hidden state.
  • Read the upgrade docs before doing it. Immutable OSes are great until you mis-configure an upgrade.

Pick this if…

  • Default new self-host k8s: Talos.
  • AWS-native hosting containers: Bottlerocket.
  • CoreOS legacy / general container host: Flatcar.
  • openSUSE shop: MicroOS.
  • Mixed workloads on bare metal (not just containers): stick with regular Linux + Ansible.

On this page