How to Fix '0/X Nodes Available: Node(s) Didn't Match Pod Topology Spread Constraints' in Kubernetes

Threat/Impact Level: CRITICAL | Exploitability/Downtime Risk: HIGH | Time to Fix: 15 mins

TL;DR

• What broke: The scheduler cannot place your pod because the topologySpreadConstraints rule cannot be satisfied — either nodes lack the required topology labels, maxSkew is too tight, or labelSelector doesn't match any running pods, making the skew calculation impossible.
• How to fix it: Relax maxSkew, switch whenUnsatisfiable from DoNotSchedule to ScheduleAnyway for non-critical workloads, verify node labels match topologyKey, and ensure labelSelector targets existing pods.
• Use our Client-Side Sandbox above to paste your failing Deployment YAML and auto-generate the corrected topologySpreadConstraints block.

The Incident (What Does the Error Mean?)

Raw scheduler event:

0/6 nodes are available: 6 node(s) didn't match pod topology spread constraints.

Or with mixed reasons:

0/6 nodes are available: 2 Insufficient cpu, 4 node(s) didn't match pod topology spread constraints.

The Kubernetes scheduler evaluated every node in the cluster and found zero valid placement targets. The pod enters Pending state indefinitely. If this is a Deployment rollout, the new ReplicaSet stalls — old pods are not terminated (RollingUpdate default), so you're not down yet, but any scale-up or node failure that kills existing pods will not recover. For a fresh Deployment with no running replicas, you have a full outage from minute zero.

The scheduler's topology spread logic works as follows: for each candidate node, it calculates the current pod count per topology domain (e.g., per zone, per node), then checks whether placing a new pod on that node would cause the skew between the most-loaded and least-loaded domain to exceed maxSkew. If every node fails this check, scheduling halts.

The Attack Vector / Blast Radius

This is a silent capacity trap. The three most common production triggers:

1. Zone imbalance after a node failure. You have 3 zones. One zone loses a node. Existing pods are now skewed 3-3-1. With maxSkew: 1, the scheduler refuses to place anything in zones A or B because it would push skew to 2. The surviving zone is already at the limit. Result: cluster-wide scheduling freeze for this workload.

2. Label mismatch on new node pools. You added a new node group (e.g., spot instances) but the nodes don't carry topology.kubernetes.io/zone or a custom label referenced in topologyKey. The scheduler sees those nodes as belonging to no domain and excludes them entirely.

3. labelSelector matches zero pods. On initial rollout, no pods are running yet. Some versions of the scheduler treat an empty match as a skew of 0 everywhere — others do not, depending on minDomains configuration. With minDomains set higher than available labeled domains, scheduling blocks immediately.

Blast radius: Every pod in the affected Deployment/StatefulSet. HPA scale-out events will queue indefinitely, burning the HPA's sync loop. In StatefulSets, a stuck pod blocks all higher-ordinal pod creation. Cluster Autoscaler will attempt to provision nodes but will not help if the constraint is a label or skew logic issue — it provisions nodes that also fail the constraint, burning cloud budget with zero scheduling relief.

How to Fix It (The Solution)

Diagnosis First

# Get the exact scheduler message
kubectl describe pod <pending-pod-name> -n <namespace> | grep -A 20 "Events:"

# Check what topology domains actually exist
kubectl get nodes --show-labels | grep topology.kubernetes.io/zone

# Count pods per domain to see current skew
kubectl get pods -n <namespace> -l <your-app-label> -o wide

Basic Fix: Relax the Constraint

The immediate unblock for production. Change whenUnsatisfiable to stop hard-blocking scheduling.

 apiVersion: apps/v1
 kind: Deployment
 spec:
   template:
     spec:
       topologySpreadConstraints:
       - maxSkew: 1
         topologyKey: topology.kubernetes.io/zone
-        whenUnsatisfiable: DoNotSchedule
+        whenUnsatisfiable: ScheduleAnyway
         labelSelector:
           matchLabels:
             app: my-service

This allows scheduling to proceed and records the violation as a best-effort, rather than blocking. Use this as the emergency lever, not the permanent config.

Enterprise Best Practice: Correct Multi-Constraint Configuration

For production workloads requiring real HA, the correct pattern combines a relaxed skew with minDomains and a node affinity guard:

 apiVersion: apps/v1
 kind: Deployment
 spec:
   template:
     spec:
+      # Ensure pods only land on nodes that actually have zone labels
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: topology.kubernetes.io/zone
+                operator: Exists
       topologySpreadConstraints:
-      - maxSkew: 1
-        topologyKey: topology.kubernetes.io/zone
-        whenUnsatisfiable: DoNotSchedule
-        labelSelector:
-          matchLabels:
-            app: my-service
+      - maxSkew: 2
+        topologyKey: topology.kubernetes.io/zone
+        whenUnsatisfiable: DoNotSchedule
+        labelSelector:
+          matchLabels:
+            app: my-service
+        # Require at least 2 zones to exist before enforcing spread
+        minDomains: 2
+      # Secondary spread across nodes within a zone
+      - maxSkew: 1
+        topologyKey: kubernetes.io/hostname
+        whenUnsatisfiable: ScheduleAnyway
+        labelSelector:
+          matchLabels:
+            app: my-service

Key decisions explained:

• maxSkew: 2 on zone spread gives the scheduler room during rolling updates and zone failures without blocking.
• minDomains: 2 prevents the constraint from activating until at least 2 zones are available — critical for single-zone dev clusters that mirror prod configs.
• The hostname-level spread with ScheduleAnyway distributes within zones as a best-effort without becoming a hard blocker.
• nodeAffinity with Exists on the zone label ensures nodes without topology labels are categorically excluded rather than silently mishandled.

💡 Tired of pasting proprietary configs into ChatGPT? Generic AI tools log your company's ARNs, DB strings, and private keys. StackEngine is a zero-backend, pure Client-Side WASM utility. Drop your failing config into the sandbox above. We redact your secrets locally in the browser and auto-generate the refactored code using your own API key.

Prevention in CI/CD

1. OPA/Gatekeeper policy — reject maxSkew: 1 with DoNotSchedule on critical namespaces:

package kubernetes.topology

violation[{"msg": msg}] {
  input.review.object.kind == "Deployment"
  constraint := input.review.object.spec.template.spec.topologySpreadConstraints[_]
  constraint.whenUnsatisfiable == "DoNotSchedule"
  constraint.maxSkew < 2
  msg := "topologySpreadConstraints with maxSkew < 2 and DoNotSchedule is fragile in multi-zone clusters. Use maxSkew >= 2 or ScheduleAnyway."
}

2. Kyverno policy — enforce minDomains presence:

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-topology-min-domains
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-min-domains
    match:
      resources:
        kinds: [Deployment]
    validate:
      message: "topologySpreadConstraints must specify minDomains when whenUnsatisfiable is DoNotSchedule"
      deny:
        conditions:
        - key: "{{ request.object.spec.template.spec.topologySpreadConstraints[].whenUnsatisfiable }}"
          operator: AnyIn
          value: ["DoNotSchedule"]

3. kubectl --dry-run in your deploy pipeline:

# Catches scheduling simulation failures before merge
kubectl apply --dry-run=server -f deployment.yaml

The --dry-run=server flag runs full admission webhook validation including scheduler predicates on the current live cluster state. Wire this into your PR pipeline as a required check against a staging cluster that mirrors prod node topology.

4. Pluto / kubeval in pre-commit: Catch deprecated topologyKey values and missing labelSelector fields before they hit the API server.