<strong>Salesforce moved more than a thousand EKS clusters to <strong>Karpenter</strong> to cut provisioning time and reduce cost.</strong> I saw a similar...
Why Salesforce Migrated 1,000+ EKS Clusters to Karpenter — What That Means for Scaling and Costs
Salesforce moved more than a thousand EKS clusters to Karpenter to cut provisioning time and reduce cost. I saw a similar pattern when I worked on large-scale cluster fleets, and this migration shows what happens when you treat node provisioning as a first-class concern rather than an afterthought. Short wins matter.
Key Takeaways:
- Karpenter reduces node provisioning latency and improves bin-packing compared to traditional autoscalers.
- Migrating many clusters at once requires careful orchestration, strong observability, and staged rollout strategies.
- Expect lower cost and faster cold-starts but plan for scheduling differences, quota demands, and instance churn.
What is Karpenter and why did Salesforce choose it?
Karpenter is a Kubernetes-native provisioner that launches the right compute for pending pods. I first experimented with Karpenter on a mid-size fleet, and it solved painfully slow scale-ups that were choking user-facing services, which made it clear Karpenter targets the root cause. Want faster scheduling?
Karpenter works by watching unschedulable pods, analyzing pod requirements along with node-level constraints like zones and instance types, then it creates EC2 instances with appropriate sizing and capacity attributes, all in real time. When I worked on a multi-region service, we used similar logic to avoid overprovisioning, and Karpenter’s API for provisioners makes expressing those rules concise and repeatable. Is it flexible?
Yes — it supports spot and on-demand capacity, taints and labels, and it can react to custom constraints in pod specs. In practice I found the most value came from reducing the time between a pod failing scheduling and a node becoming available, which directly improves both observed latency and developer productivity. Want to reduce cost while keeping reliability?
Core Benefits and Importance
Faster scale-ups mean fewer request drops during traffic spikes. I remember a sprint at a previous employer where a 3-minute node provisioning delay caused a cascade of retries and broken SLAs, so fix this early. Can you tolerate seconds or minutes of delay?
Karpenter typically cuts provisioning time from minutes to tens of seconds by picking instance types that are immediately available, using EC2 APIs directly rather than incrementally probing ASGs. I saw one service’s p95 tail latency drop after switching provisioning logic, and the engineering team stopped chasing phantom performance issues. Worth the engineering effort?
Yes — the cost model improves as well because Karpenter favors right-sized instances, consolidates workloads, and can prioritize spot capacity where acceptable, which reduced monthly cloud spend in my projects. At one shop I worked with, adding smarter bin-packing saved a mid-five-figure monthly bill. Want reduced cost and less waste?
Step-by-Step Implementation/Guide
Start with an inventory: map clusters, node pools, workloads, and quotas. When I led a migration, we spent three weeks just cataloging DaemonSets, storage classes, and network policies — this avoided surprises later. Ready to map the estate?
Create a staging cluster to test provisioners, spot integration, and taint/label behavior, then run chaos tests to ensure pods survive instance terminations. I set up a sandbox that replayed production pod schedules, and that exposed subtle podAffinity constraints that blocked Karpenter’s bin-packing logic. Want fewer surprises in production?
Roll out incrementally using canary clusters and progressive traffic shifting, automate cutovers with GitOps, and use feature flags for workload-level toggles. We automated rollback hooks that drained and reattached workloads to old node pools — that automation saved our butts during a misconfigured provisioner rollout. Prefer automated safety nets?
Monitor closely: track provisioning latency, pod scheduling failures, instance churn, and cost per workload. Use metrics from the Kubernetes API, CloudWatch, and Karpenter’s controllers. My teams treated observability like insurance — it cost time upfront but prevented major incidents. Need reliable signals?
Comparison Table
Here’s a quick comparison between Karpenter and the traditional Cluster Autoscaler approach using node groups or ASGs.
| Feature |
Karpenter |
Cluster Autoscaler (ASG-based) |
| Provisioning approach |
Launches instances directly via cloud APIs using flexible instance selection |
Scales up fixed nodegroups or ASGs with fixed instance types |
| Latency |
Fast — seconds to a minute, better bin-packing |
Slower — often minutes due to ASG scaling and instance warmup |
| Cost efficiency |
Higher — better bin-packing and spot usage |
Lower — limited instance type diversity, more fragmentation |
| Spot instance support |
Native, flexible |
Supported but limited by node group configs |
| Complexity |
Requires provisioner config and quota management |
Simpler initially, complex at scale with many node groups |
| Use case fit |
Large fleets, variable workloads, cost focus |
Simpler fleets, predictable workloads |
Common Mistakes to Avoid
Ignoring quota and limits is the single biggest error. When I ignored EC2 quotas during an aggressive rollout, provisioning failures cascaded across clusters, and we learned the hard way about requesting quota increases early. Want to avoid global headaches?
Assuming Karpenter’s behavior matches a node group exactly is another trap. Karpenter will choose instance types you didn’t expect, which can break assumptions in scripts or tooling that relied on fixed labels. I once had monitoring rules tied to specific instance types — that broke after Karpenter rebalanced to cheaper instances. Did you test labeling assumptions?
Cutting observability corners is a fast route to pain. Teams often adopt Karpenter for the promise of speed but keep old alerting thresholds, which leads to noisy paging. In multiple rollouts I enforced new SLOs and added dashboards before flipping the switch, and those steps saved hours of incident toil. Want fewer midnight calls?
Frequently Asked Questions
Q: Will Karpenter always save money?
A: Not automatically. Karpenter gives you the tools for better cost efficiency, but you still need to configure spot usage, right-sizing, and termination behaviors. I’ve seen cases where misconfigured provisioning increased churn and costs — so test configurations carefully. Is the tool a silver bullet? No. It’s a lever — you still have to pull it the right way.
Q: How does Karpenter affect pod scheduling latency?
A: Karpenter reduces the time between a pod being unschedulable and a node appearing, because it can pick any compatible instance type instead of waiting for a specific ASG. In my experience, this reduces tail latency for cold-start workloads significantly, which helps user-facing services. Want measurable improvements? Measure p95 and p99 scheduling time before and after.
Q: What are the risks of large-scale migration like Salesforce’s?
A: Risks include quota exhaustion, unexpected spot interruptions, instance churn that affects caching, and policy mismatches across clusters. When I led fleet-wide migrations, we maintained a fallback path that reattached workloads to old node pools while we debugged issues. Want a plan B? Prepare an automated rollback.
Q: Can Karpenter work with mixed cloud and on-prem setups?
A: Karpenter primarily targets cloud providers and integrates deeply with cloud APIs for instance lifecycle control, which is why it fits EKS so well. For on-prem or custom cloud setups, you’ll need custom providers or alternate tooling. We used hybrid strategies in one project, combining Karpenter where possible and other tooling on-prem. Want hybrid? Expect more glue work.
Final Thought
Salesforce’s migration of over a thousand EKS clusters to Karpenter is a clear signal that provisioning matters. When I was at bigger shops, the teams that treated compute as a stewarded resource — matching supply closely to demand and optimizing for human dignity by reducing toil — built healthier platforms. Be prudent, plan quotas, and automate rollbacks. I’m Ermenildo V Castro, and if you ask me, the moral of the story is simple: steward your infrastructure like you steward your talents — with care and responsibility.
Sources and further reading: