Running Talos Linux in production: a field report

If you want one server you can never break by logging into it and fat-fingering a config, Talos Linux is the closest thing I have run. After about five months of running it in production — since early 2026, across a cluster I rebuilt from the OS up — my verdict is simple: the no-SSH, declarative model is correct, the upgrade scheme is the safest I have used, and the price you pay is that every old Linux reflex you have — ssh in, edit a file, restart a service — is now an error. That last part is the whole experience, and it took me a while to stop resenting it.

I mentioned in Building My Own Cloud that I run Talos on every node. This is the longer version: what it is, how you actually stand a cluster up, what the upgrade model really does, and the specific things that made me swear at two in the morning. It is a field report, not a tutorial — I am telling you what it was like, with the commands that matter, so you can decide whether the trade is yours to make.

What Talos actually is

Talos Linux is a Linux distribution built to do exactly one thing: run Kubernetes. Its own docs put it plainly — “It does one thing and it does it better than any general purpose Linux distribution” (Sidero Labs, What is Talos, accessed Jun 2026). Everything that makes it strange follows from that single focus.

There is no SSH daemon. No shell. No package manager. No systemd you log in to poke. The root filesystem is immutable. The entire machine — network, disks, kernel arguments, every Kubernetes component flag — is described in one YAML document called the machine config, and the only way to talk to the node is a single declarative gRPC API, authenticated with mutual TLS (Sidero Labs, What is Talos, accessed Jun 2026). The client for that API is talosctl. You do not administer a Talos box; you reconcile it.

If you have spent fifteen years in mutable Linux — and I had — this is disorienting before it is liberating. The thing that finally landed for me was realising that almost every production incident I had ever caused was a human typing into a shell on a box that mattered. Talos removes the shell. It removes the category of mistake. The cost is that it also removes the category of quick fix, and you have to make your peace with that before you will be happy here.

The current stable line, for grounding, is v1.13.x — v1.13.4 shipped 9 June 2026 on Linux 6.18.34 with etcd v3.6.12 (siderolabs/talos releases, accessed Jun 2026). I am running v1.13.3.

The install: gen config, apply-config, bootstrap

The bare-metal install is four commands. Not four scripts — four talosctl invocations. This is the part the documentation gets genuinely right, and it is worth walking through because it tells you everything about the model.

First you generate the cluster’s config and PKI:

talosctl gen config <cluster-name> <cluster-endpoint>

This produces controlplane.yaml, worker.yaml, and a talosconfig client file (Sidero Labs, Getting Started, accessed Jun 2026). The two YAML files are the machine configs. The talosconfig is your client credential — guard it the way you guard a root key, because functionally it is one.

You boot each node off the Talos image. It comes up in maintenance mode — alive on the network, no cluster, waiting for a config. You push the config to it over the API, and because there is no PKI established yet, you do it insecurely on purpose:

talosctl apply-config --insecure --nodes 192.168.0.2 --file controlplane.yaml

That --insecure flag is a one-time thing for the maintenance-mode handshake (Sidero Labs, Getting Started, accessed Jun 2026). Once the node has its config, mTLS is on and every subsequent call is authenticated.

Then — and this is the step that bites people — you bootstrap etcd. Exactly once. On exactly one control plane node:

talosctl bootstrap --nodes 192.168.0.2

The docs are emphatic, in their own capital letters: “The bootstrap operation should only be called ONCE on a SINGLE control plane node” (Sidero Labs, Getting Started, accessed Jun 2026). It does not matter that you have three control plane nodes. You bootstrap one. The others join the etcd quorum on their own. Run bootstrap twice and you will have a bad day explaining to yourself why etcd is confused about who is in the cluster.

Last, you pull a kubeconfig and you have a working cluster:

talosctl kubeconfig

That is it. Bare metal to a running Kubernetes cluster in four commands, all of them declarative, none of them involving a login. Sidero’s own marketing claims “bare metal to Kubernetes in under a minute” (vendor: Sidero Labs) and my own experience is that the shape of the claim is honest — my wrong turns took the time, not the machinery. The steps are genuinely few and they run fast.

The no-SSH model in daily life

Here is what nobody tells you up front: the absence of SSH changes how you think, not just how you operate.

On a normal Linux box, when something is wrong, your hands move before your brain does — ssh, tail -f, vim, restart. On Talos that muscle memory is dead weight. There is no door to go through. Instead you ask the API questions. talosctl logs. talosctl dmesg. talosctl services. talosctl get <resource> to read the machine’s actual reconciled state. You are not inside the box looking around; you are interrogating it from outside through a typed API.

At first this felt like fighting with one hand tied. By the time I had stood the cluster up and lived in it for a while, it felt like the correct amount of hands. Every question I could ask had a structured answer, and — crucially — nothing I asked could change the machine by accident. Reading state and mutating state are different verbs in Talos. On mutable Linux they are dangerously close: you open a file to look, and now you are one keystroke from editing it.

To change anything, you edit the machine config and apply it:

talosctl apply-config --nodes 192.168.0.2 --file controlplane.yaml

No --insecure this time — the node is past maintenance mode, so it is authenticated. The node reconciles to the new desired state. If the change needs a reboot, Talos tells you, or you stage it. The config is a file. It lives in git. The cluster is, genuinely and not as a slogan, the contents of a few YAML documents you can diff. The drift problem — the slow divergence between “what the runbook says” and “what someone actually did to the box at 3am” — does not exist, because there is no mechanism by which someone could have done something to the box that is not in the file.

That is the single best property of the system and I would not give it up.

The upgrade model is the best part

I have upgraded a lot of operating systems in production and lived through the consequences. Talos has the only OS upgrade model I actually trust, and it comes down to one design choice: an A/B image scheme.

“Upgrades use an A-B image scheme in order to facilitate rollbacks. This scheme retains the previous Talos kernel and OS image following each upgrade” (Sidero Labs, Upgrading Talos Linux, accessed Jun 2026). The boot partition holds two slots. One is live. The upgrade writes the new image to the inactive slot and reboots into it, leaving the version you were running untouched in the other slot.

You trigger it by naming the installer image for the version you want:

talosctl upgrade --nodes 10.20.30.40 \
  --image ghcr.io/siderolabs/installer:v1.9.5

(Sidero Labs, Upgrading Talos Linux, accessed Jun 2026.)

Now the part that converts skeptics. If the new image fails to boot, you do nothing. “The bootloader will automatically use the previous Talos kernel and image, thus effectively rolling back the upgrade” (Sidero Labs, Upgrading Talos Linux, accessed Jun 2026). The failure mode of an OS upgrade — the one that has cost me whole nights elsewhere — is a box that does not come back. Talos’s answer is that the box comes back on the old version, by itself, because the old version never left. There is a manual talosctl rollback too if you decide a booting upgrade was still a mistake.

Two flags I have learned to respect. --stage writes the upgrade artifacts to disk and applies them on the next reboot, very early in boot, before anything holds files open — Talos describes it as putting the artifacts and some metadata on a partition “that gets checked very early in the boot process” so there is “no conflict with any files being held open” (Sidero Labs, Upgrading Talos Linux, accessed Jun 2026). And on control plane nodes, Talos protects you from yourself: “Talos will refuse to upgrade a control plane node if that upgrade would cause a loss of quorum for etcd,” and if you ask several control plane nodes to upgrade at once, it serialises them so only one upgrades at a time while it checks quorum (Sidero Labs, Upgrading Talos Linux, accessed Jun 2026).

Upgrading the OS and upgrading Kubernetes are deliberately separate. The Talos version and the Kubernetes version are different things on different clocks. Kubernetes gets its own command:

talosctl upgrade-k8s --nodes <node> --to <k8s-version>

It walks the control plane components and kubelets in the right order to keep the cluster available, and it takes --dry-run so you can see the plan before you commit (Sidero Labs docs, accessed Jun 2026). Separating the two is the right call. The number of times the OS and the orchestrator should be forced to move together is zero, and Talos does not force it.

What’s genuinely painful

I would not be honest — and honesty is the whole contract on this blog — if I made this sound frictionless. It is not. Here is the real list.

Your debugging instincts are wrong on day one. Everything you know about diagnosing a Linux box assumes a shell. When a node misbehaves, you cannot drop in and strace a thing. You learn the talosctl equivalents, and there is a real equivalent for almost everything, but “almost” is doing work in that sentence. The case that made me miss the shell badly was a storage stall on the Hetzner vSwitch: replicated storage between two nodes showed the connection as fine, yet resync was frozen. On a normal box my reflex would have been to SSH in and tcpdump the interface until I saw the dropped frames. On Talos I had to chase it from outside — and the answer was that the vSwitch silently drops full-size frames, so small packets passed and the connection looked healthy while large ones vanished. Pinning the interface MTU down to 1400 fixed it. The diagnosis was harder without a shell to live in; the discipline of doing it through the API is also why the fix is now a line in a config file instead of a thing I did to a box at night.

The machine config is unforgiving and the surface is large. Everything-in-one-YAML is a strength right up until you get a key wrong, and then you are reading schema docs to find out whether the field lives under machine: or cluster: and which version of the config it belongs to. The config has its own versioning, and following an old guide against a newer Talos is a reliable way to waste an afternoon.

Bare-metal networking and disks are your problem. Talos boots fast and reconciles cleanly, but it does not paper over a NIC that needs a non-obvious driver, a disk selector that matches the wrong device, or a bonded interface you configured slightly wrong. On a cloud image these are abstracted away. On bare metal they are yours, and Talos will faithfully apply exactly the wrong config you handed it.

You cannot cowboy. This is a strength I am filing under pain because of how it feels in the moment. There is no “just SSH in and fix it for now, do it properly later.” Every fix is a config change applied through the API. Ninety-five percent of the time this is correct and you are grateful. The other five percent — the genuine emergency where you want raw access to a single box and the discipline is in your way — you will feel the walls. The walls are the point. It still chafes.

The learning curve is front-loaded. The first cluster is harder than it should be, because you are unlearning as much as learning. Past that hump it gets easier in a way mutable Linux never does — there is no entropy accumulating on the boxes, no slow rot of hand-applied changes. But the hump is real and you should budget for it. Mine was front-loaded into the first cluster build; by the second rebuild the model was reflex rather than friction.

Would I run it again

Yes, without hesitation, for this kind of workload — bare metal, Kubernetes, a small number of nodes I want to treat as cattle and never as pets.

The trade is clean once you see it. You give up the shell and the quick fix. You get an OS that cannot drift, cannot be broken by a careless login, and rolls itself back when an upgrade goes wrong. For a substrate I want to forget about between the times I am deliberately changing it, that is exactly the trade I want.

I would not reach for it if I needed a general-purpose box that happened to also run some containers, or if SSH-based break-glass access were a hard requirement for my context, or if my team’s whole muscle memory was mutable-Linux and the retraining cost outweighed the workload. Talos is not a friendlier Ubuntu. It is a different model with a different cost structure, and the cost is mostly paid up front in unlearning.

I paid that cost. I would pay it again. The machine I cannot break by logging into it turned out to be the machine I most wanted to own.