Karpenter Spot Instance Capacity Debugger

Threat/Impact Level: HIGH | Downtime Risk: HIGH | Time to Fix: 15–30 mins

TL;DR

• What broke: Karpenter cannot find any EC2 Spot capacity matching your NodePool's instance type constraints in the requested Availability Zone(s), leaving pods in Pending indefinitely.
• How to fix it: Broaden instance family diversity, add multi-AZ spread, and configure an on-demand fallback weight so Karpenter has an escape hatch when Spot pools dry up.
• Shortcut: Use our Client-Side Sandbox above to paste your NodePool YAML and controller logs — it will auto-refactor the instance requirements and capacity fallback config without sending your data anywhere.

ERROR   controller.provisioner   no instance types available   {"commit": "abc1234", "provisioner": "default"}
ERROR   controller.nodeclaim     failed to launch nodeclaim    {"error": "no capacity available for node request"}

kubectl get pods -A | grep Pending
my-namespace   worker-7d9f6   0/1   Pending   0   18m

How to Fix It

Basic Fix — Broaden Instance Diversity

The minimum viable fix is to stop pinning to a single instance family and let Karpenter use karpenter.k8s.aws/instance-category with multiple families.

apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
-       - key: node.kubernetes.io/instance-type
-         operator: In
-         values: ["m5.xlarge"]
-       - key: topology.kubernetes.io/zone
-         operator: In
-         values: ["us-east-1b"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot"]
+       - key: karpenter.k8s.aws/instance-category
+         operator: In
+         values: ["m", "c", "r"]
+       - key: karpenter.k8s.aws/instance-generation
+         operator: Gt
+         values: ["4"]
+       - key: topology.kubernetes.io/zone
+         operator: In
+         values: ["us-east-1a", "us-east-1b", "us-east-1c"]
+       - key: karpenter.sh/capacity-type
+         operator: In
+         values: ["spot", "on-demand"]

Enterprise Best Practice — Weighted On-Demand Fallback + Disruption Budgets

For production, you need two NodePools: one Spot-optimized, one On-Demand fallback with a higher weight so Karpenter prefers Spot but can always fall back. Combine with disruption budgets to prevent thrashing during Spot reclamation events.

# NodePool 1: Spot (preferred)
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: spot-workers
spec:
+ weight: 100
  disruption:
+   consolidationPolicy: WhenUnderutilized
+   consolidateAfter: 30s
+   budgets:
+   - nodes: "20%"  # Never consolidate more than 20% of nodes at once
  template:
    spec:
      requirements:
+       - key: karpenter.sh/capacity-type
+         operator: In
+         values: ["spot"]
+       - key: karpenter.k8s.aws/instance-category
+         operator: In
+         values: ["m", "c", "r"]
+       - key: karpenter.k8s.aws/instance-generation
+         operator: Gt
+         values: ["4"]
+       - key: topology.kubernetes.io/zone
+         operator: In
+         values: ["us-east-1a", "us-east-1b", "us-east-1c"]
---
# NodePool 2: On-Demand fallback
apiVersion: karpenter.sh/v1beta1
kind: NodePool
metadata:
  name: ondemand-fallback
spec:
+ weight: 1  # Only used when spot-workers NodePool cannot schedule
  template:
    spec:
      requirements:
+       - key: karpenter.sh/capacity-type
+         operator: In
+         values: ["on-demand"]
+       - key: karpenter.k8s.aws/instance-category
+         operator: In
+         values: ["m", "c"]
+       - key: topology.kubernetes.io/zone
+         operator: In
+         values: ["us-east-1a", "us-east-1b", "us-east-1c"]

Also verify your EC2NodeClass has correct amiSelectorTerms and subnetSelectorTerms — a misconfigured NodeClass silently eliminates valid instance types before Karpenter even evaluates capacity:

apiVersion: karpenter.k8s.aws/v1beta1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: "my-cluster"  # Must match actual subnet tags
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: "my-cluster"

💡 Tired of pasting proprietary infrastructure configs into public LLMs? Generic AI interfaces log your company's ARNs, connection strings, and private keys. StackEngine enforces client-side zero-trust boundaries—the sandbox above executes 100% locally in your browser. Drop your broken manifest above to redact secrets locally and synthesize deterministic patch drafts with your own API key.

(For CI/CD pipeline automation, install via npm i -g stackengine).

package karpenter.nodepool

deny[msg] {
  input.kind == "NodePool"
  reqs := input.spec.template.spec.requirements
  zone_req := reqs[_]
  zone_req.key == "topology.kubernetes.io/zone"
  zone_req.operator == "In"
  count(zone_req.values) < 2
  msg := "NodePool must span at least 2 Availability Zones to tolerate Spot capacity outages."
}

deny[msg] {
  input.kind == "NodePool"
  reqs := input.spec.template.spec.requirements
  type_req := reqs[_]
  type_req.key == "node.kubernetes.io/instance-type"
  type_req.operator == "In"
  count(type_req.values) < 3
  msg := "NodePool pinned to fewer than 3 instance types. Spot interruptions will cause scheduling failures."
}

# checkov custom check: CKV_KARPENTER_001
# Validates NodePool has capacity-type including on-demand fallback OR weight-based fallback pool exists

checkov -d ./helm/karpenter --framework kubernetes --check CKV_KARPENTER_001

aws cloudwatch put-metric-alarm \
  --alarm-name "karpenter-pending-pods-breach" \
  --metric-name "pending_pods" \
  --namespace "ContainerInsights" \
  --statistic Average \
  --period 300 \
  --threshold 1 \
  --comparison-operator GreaterThanOrEqualToThreshold \
  --evaluation-periods 1 \
  --alarm-actions arn:aws:sns:us-east-1:123456789:oncall-pagerduty

# In your Karpenter Helm values
interruptionQueue: "my-cluster-karpenter-interruption-queue"