Calico IPv6 Dual-Stack IPAM Debugger

The Incident (What does the error mean?)

Raw log output from a crashing calico-node pod:

E1105 03:12:44.882341       1 ipam.go:312] Failed to get IP address from pool: no IPv6 pools found
E1105 03:12:44.882411       1 main.go:198] Error: failed to get IP from pool: no IPv6 pools found for node calico-node-xk9p2
FATAL calico/node: failed to start: failed to get IP from pool

Immediate consequence: Every calico-node pod on nodes that were assigned an IPv6 address during kubelet registration enters CrashLoopBackOff. The node never becomes Ready. Any workload pod scheduled to that node stays in ContainerCreating indefinitely. In a rolling upgrade or a fresh node-pool scale-out, this silently bricks your entire new node capacity.

The Attack Vector / Blast Radius

This is a scheduling dead-zone failure. The blast radius scales with your node count:

• Node-level: calico-node DaemonSet pod crashes → node CNI is non-functional → kubelet reports NetworkPlugin not ready → node taints itself node.kubernetes.io/not-ready.
• Workload-level: All pods scheduled to the affected node are blocked. Existing pods are NOT evicted immediately, but any restart (OOMKill, rolling deploy, HPA scale-up) on that node will fail to re-attach networking.
• Control-plane ripple: If the node running coredns or a critical system pod is affected, DNS resolution cluster-wide degrades.
• Dual-stack-specific trap: You enabled dual-stack (--cluster-cidr=10.244.0.0/16,fd00::/48) in kube-controller-manager and kubelet, but Calico's IPAM is independent of Kubernetes IPAM. Calico requires its own IPPool CRD for each address family. Kubernetes knowing about your IPv6 CIDR means nothing to calicoctl.

The most common triggers in production:

1. Dual-stack enabled on the cluster after initial Calico install — the IPv6 IPPool was never created.
2. blockSize set to /128 (single address per block) causing immediate exhaustion.
3. ipipMode: Always set on the IPv6 pool — IPIP is IPv4-only; Calico silently marks the pool invalid.
4. disabled: true left in the IPPool manifest from a staging copy-paste.

Prevention in CI/CD

1. OPA/Gatekeeper policy — block IPPool with ipipMode on IPv6

package calico.ippool

deny[msg] {
  input.kind == "IPPool"
  contains(input.spec.cidr, ":")
  input.spec.ipipMode != "Never"
  msg := sprintf("IPPool %v: ipipMode must be 'Never' for IPv6 CIDRs", [input.metadata.name])
}

deny[msg] {
  input.kind == "IPPool"
  contains(input.spec.cidr, ":")
  input.spec.disabled == true
  msg := sprintf("IPPool %v: IPv6 pool must not be disabled in production", [input.metadata.name])
}

2. Checkov custom check (Python)

# checkov/custom/check_calico_ipv6_pool.py
from checkov.common.models.enums import CheckResult, CheckCategories
from checkov.kubernetes.checks.resource.base_resource_check import BaseK8Check

class CalicoIPv6PoolCheck(BaseK8Check):
    def __init__(self):
        name = "Ensure Calico IPv6 IPPool has ipipMode=Never and is not disabled"
        id = "CKV_K8S_CALICO_001"
        super().__init__(name=name, id=id,
                         categories=[CheckCategories.NETWORKING],
                         supported_entities=["IPPool"])

    def scan_resource_conf(self, conf):
        cidr = conf.get("spec", {}).get("cidr", "")
        if ":" not in cidr:
            return CheckResult.PASSED
        spec = conf.get("spec", {})
        if spec.get("ipipMode", "Never") != "Never":
            return CheckResult.FAILED
        if spec.get("disabled", False):
            return CheckResult.FAILED
        return CheckResult.PASSED

3. Pre-flight in your Helm/Terraform pipeline

# Add to your cluster bootstrap script BEFORE calico-node DaemonSet is applied
IPV6_POOL=$(calicoctl get ippools -o json | jq '[.items[] | select(.spec.cidr | contains(":"))] | length')
if [ "$IPV6_POOL" -eq 0 ]; then
  echo "FATAL: No IPv6 IPPool found. Apply ipv6-ippool.yaml before proceeding."
  exit 1
fi

This pre-flight gate catches the missing pool before calico-node is rolled out, eliminating the CrashLoop entirely in automated provisioning pipelines.