KubeVirt VMI Virt-Launcher Crash Monitor

Threat/Impact Level: HIGH | Downtime Risk: HIGH | Time to Fix: 15–45 mins depending on root cause

TL;DR

• What broke: The virt-launcher pod for your KubeVirt VirtualMachineInstance crashed or was OOMKilled, leaving the VMI in a perpetual Scheduling or Pending phase with VMI not ready surfaced by the virt-controller.
• How to fix it: Identify the crash reason via kubectl logs and kubectl describe on the virt-launcher pod — fix is one of: bump resource limits, patch the VMI spec's CPU/memory, correct the PVC binding, or resolve a libvirt/QEMU socket permission issue.
• Shortcut: Use our Client-Side Sandbox above to auto-refactor your failing VMI YAML — paste it in, get the corrected spec back without sending your config to a third-party server.

# kubectl get vmi <vmi-name> -n <namespace>
NAME         AGE   PHASE     IP    NODENAME   READY
my-vm-0      4m    Scheduling                  False

# kubectl describe vmi my-vm-0 -n <namespace>
...
Warning  SyncFailed   3m   virt-controller  Error creating virt-launcher pod: ...
Warning  VMINotReady  2m   virt-handler     VMI not ready: virt-launcher pod is not running

# kubectl describe pod virt-launcher-my-vm-0-<hash> -n <namespace>
  Last State: Terminated
    Reason:   OOMKilled (or Error, CrashLoopBackOff)
    Exit Code: 137

How to Fix It (The Solution)

Step 1 — Pull the actual crash reason

# Get the terminated container logs (not the running one)
kubectl logs virt-launcher-my-vm-0-<hash> -n <namespace> --previous -c compute

# Also check virt-handler on the affected node
kubectl logs -n kubevirt -l kubevirt.io=virt-handler --field-selector spec.nodeName=<node>

Basic Fix — OOMKill: Raise Memory Limits on the VMI

 apiVersion: kubevirt.io/v1
 kind: VirtualMachineInstance
 metadata:
   name: my-vm-0
 spec:
   domain:
     resources:
       requests:
-        memory: 64M
+        memory: 2Gi
       limits:
-        memory: 128M
+        memory: 2Gi
     cpu:
       cores: 1

Rule: virt-launcher itself needs overhead on top of the guest RAM. KubeVirt adds an overhead buffer automatically, but if you hard-cap limits.memory below requests.memory + overhead, the pod is killed before QEMU initializes. Never set memory limits below 1Gi for any non-trivial guest.

Basic Fix — Unbound PVC

kubectl get pvc -n <namespace>
# If STATUS = Pending, your StorageClass is not provisioning
kubectl describe pvc <pvc-name> -n <namespace>

 volumes:
   - name: rootdisk
     persistentVolumeClaim:
-      claimName: my-vm-pvc-typo
+      claimName: my-vm-rootdisk-pvc

Enterprise Best Practice — Enforce Resource Floors with LimitRange + VMI Validation Webhook

Problem: Individual teams submit VMI specs with dangerously low memory. No guardrails exist at admission time.

Solution: Deploy a LimitRange in each VM namespace AND enforce a KubeVirt ValidatingWebhookConfiguration (or OPA/Gatekeeper policy) that rejects VMIs below safe thresholds.

# LimitRange for VM namespaces
 apiVersion: v1
 kind: LimitRange
 metadata:
   name: kubevirt-vm-limits
   namespace: vm-workloads
 spec:
   limits:
-  # No limits defined — any value accepted
+  - type: Container
+    min:
+      memory: 512Mi
+      cpu: "250m"
+    default:
+      memory: 2Gi
+      cpu: "1"
+    defaultRequest:
+      memory: 2Gi
+      cpu: "1"

# OPA Rego — Gatekeeper ConstraintTemplate excerpt
- # No policy — virt-launcher OOMKills silently
+ deny[msg] {
+   input.review.object.kind == "VirtualMachineInstance"
+   mem := input.review.object.spec.domain.resources.requests.memory
+   to_number(trim_suffix(mem, "Mi")) < 512
+   msg := "VMI memory request below 512Mi — virt-launcher will OOMKill"
+ }

Fix — KVM Not Available on Node

# On the suspect node
ls -la /dev/kvm
# If missing: nested virtualization is disabled in the hypervisor or BIOS

# For AWS (enable nested virt on metal instances only — .metal instance types)
# For GKE, use n2 or c2 node pools with --enable-nested-virtualization

# Verify KubeVirt node labels
kubectl get nodes -l kubevirt.io/schedulable=true

# If your VMI was scheduled to a non-KVM node due to missing nodeSelector:
 spec:
+  nodeSelector:
+    kubevirt.io/schedulable: "true"
   domain:
     ...

💡 Tired of pasting proprietary infrastructure configs into public LLMs? Generic AI interfaces log your company's ARNs, connection strings, and private keys. StackEngine enforces client-side zero-trust boundaries—the sandbox above executes 100% locally in your browser. Drop your broken manifest above to redact secrets locally and synthesize deterministic patch drafts with your own API key.

(For CI/CD pipeline automation, install via npm i -g stackengine).

pip install checkov
checkov -f vmi-manifest.yaml --framework kubernetes
# Custom check: flag any VMI with memory request < 512Mi

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-kvm-node-selector
spec:
  validationFailureAction: enforce
  rules:
    - name: check-kvm-nodeselector
      match:
        resources:
          kinds: [VirtualMachineInstance]
      validate:
        message: "VMI must target kubevirt.io/schedulable=true nodes"
        pattern:
          spec:
            nodeSelector:
              kubevirt.io/schedulable: "true"

#!/bin/bash
# pre-commit: validate PVC names in VMI specs match existing PVCs
PVC_NAMES=$(kubectl get pvc -n "$NAMESPACE" -o jsonpath='{.items[*].metadata.name}')
REF=$(yq e '.spec.volumes[].persistentVolumeClaim.claimName' vmi-manifest.yaml)
if [[ ! " $PVC_NAMES " =~ " $REF " ]]; then
  echo "ERROR: PVC '$REF' not found in namespace $NAMESPACE"
  exit 1
fi

- alert: KubeVirtVMINotReady
  expr: kubevirt_vmi_phase_count{phase="Scheduling"} > 0
  for: 5m
  labels:
    severity: critical
  annotations:
    summary: "VMI stuck in Scheduling phase for >5min — likely virt-launcher crash"
    runbook: "https://your-wiki/kubevirt-virt-launcher-crash"