It started with a postmortem I never wanted to write. A merchant flash sale launched 20 minutes ahead of schedule. Traffic to the payment authorization services doubled in under a minute. Kubernetes HPA did exactly what it was configured to do — it detected the CPU spike and requested scale-out across over 150 checkout-path services simultaneously. Most new pods fit on existing nodes, but dozens of services exhausted their node pool headroom and triggered provisioner requests in a burst. The node provisioner stalled under the queue pressure. New capacity came up four minutes later. ...
Three ways that conventional per-service autoscaling breaks down at 100,000 pods, ~1,550 applications, and five shared domains, and why no amount of HPA tuning makes them go away. Failure Mode 1: The Provisioner Stampede HPA is designed to be autonomous. Each deployment has its own HPA object, its own target utilization, its own scale-out logic. This is great for isolation. Changes to one service’s autoscaling config do not affect others. It breaks down under coordination pressure. ...
Commander and Soldiers: Decomposing the Scaling Problem Part 3 · Series: Teaching Kubernetes to Think Ahead By Aashish Sheshadri — Platform Architecture The design space for RL-based autoscaling has three obvious options. Two of them do not hold up under scrutiny. Option 1: One Big Agent The most natural first thought: replace all 1,500 HPAs with a single RL agent that observes the entire cluster and outputs replica counts for every service. ...
The previous post decomposed the scaling problem into a Commander (budget allocation across domains) and Soldiers (per-domain replica execution). The Commander’s proactive character — the thing that separates HiRL-Scale from “reactive RL that is faster than HPA” — comes from a single component: the traffic forecaster. I use a Temporal Fusion Transformer (TFT) to produce per-domain RPS forecasts at 5, 15, and 30-minute horizons. I feed the 80th-percentile forecast into the Commander’s observation. I train the forecaster offline and freeze it during RL training. Every one of those choices bit me. ...
The previous post ended on a claim I owe you evidence for: that getting the reward function right was the hardest part of this project. Before I can show you that, I have to show you what the Commander is actually working with — what it can see, and what it is allowed to do about it. Those two things bound everything the reward function can possibly ask for. The Commander never directly touches a replica count. It does not know how many pods service X is running. All it does is answer one question: how should I distribute the cluster’s capacity across five domains, right now, given what I know and what I expect? ...
The previous post laid out what the Commander can see and what it is allowed to do. This one is about the part that decides what it should do, which is where the design decisions matter most and where I made nearly every mistake worth writing down. Everything below — every failure mode, every reward-hacking episode, every behavioral property — was observed in simulation during offline training. None of it happened on a production cluster. That is by construction: at a 60-second tick, the roughly five million steps this policy needed to converge would take about nine and a half years of wall-clock time to collect from the real thing. Training in production is not inadvisable here, it is arithmetically impossible. Shadow mode on production contributes under one percent of what the policy knows, and none of that percent is on-policy. ...
The previous post was about writing a reward function and watching a policy exploit it. This one is about the harder question underneath: how did I know any of it was working? Not “did the loss go down.” Loss went down the entire time, including during the weeks the policy was quietly learning to game me. I mean the actual question — is this thing doing what I meant, and how would I find out if it were not. ...