PostEngineering leadership

A rollback that restores the old configuration may still leave the system broken.

Sessions, caches, routing state and partial side effects do not revert because the file did — which makes 'previous version available' and 'operation reversible' different properties.

Lukman Nuriakhmetov

1 min read · August 5, 2026

State does not roll back because the file did.

I have watched routine changes cause disproportionate incidents: a firewall rule, DNS shift, certificate renewal, or failover that looked harmless in isolation.

The configuration was reverted, but the service did not recover. Sessions had been dropped. Caches were stale. Routing state had diverged. A downstream system had already accepted part of the change. Agentic workflows add the same problem at another layer: tool calls, retries, durable tasks, and external actions can leave partial state behind even when the agent itself stops.

Rollback design has to identify what is restored, invalidated, replayed, or compensated.

"Previous version available" and "operation reversible" are not the same property.

Tags: systems-thinking · reliability · engineering-leadership · rollback-design

Explore this topic

Engineering leadership →

Related thinking

NoteAugust 24, 2026

The Model Stopped Being the Interesting Variable

Week of August 17, 2026. Mistral tripled accuracy on financial filings — 26.7% to 86% — by changing how the model searches, not which model searches. Two different models from two different labs got the same roughly threefold lift from the same harness. That pattern ran through the week: authorisation splitting into four separate questions, a token optimisation that cut tokens and raised total cost by half, throughput outgrowing the review capacity around it, and execution moving into shared team channels. The leverage kept turning out to sit outside the model. A scan-layer TL;DR, five themes each with an operator move, one taken deep, three counter-signals, and what to track.

Weekly Read

14 min

NoteAugust 17, 2026

The Consequence Detached From the Cause

Week of August 10, 2026. A compromise from March turned out this month to have touched roughly 2,500 organisations — the blast radius took five months to become visible. That gap ran through the whole week: authority outliving the context that granted it, one upstream assumption fanning out across a graph of agents, a fast step producing no faster system, a metric drifting from the thing it measures. The link between an action and its consequence stopped being local. A scan-layer TL;DR, five themes each with an operator move, one taken deep, three counter-signals, and what to track.

Weekly Read

14 min

NoteAugust 10, 2026

Capability Outran Confirmation

Week of August 3, 2026. Frontier models generated 6,080 security patches and roughly a quarter of them worked — while a plausible-but-wrong hint in the prompt cut the success rate to one in six, and the agents followed the hint over their own evidence. The same gap opened everywhere: peak capability diverging from production reliability, agents that cannot verify themselves, operational data capping what any model can automate, and portable artifacts carrying no portable trust. Production got cheap. Confirming what was produced did not. A scan-layer TL;DR, five themes each with an operator move, one taken deep, three counter-signals, and what to track.

Weekly Read

14 min