SE
STACKENGINE
Zero-Trust Client IDE/ Security

SealedSecrets Unseal Private Key Verifier

SealedSecrets controller cannot decrypt sealed secrets because the cluster's private key no longer matches the key used during encryption, causing all dependent workloads to fail secret injection.

SYSTEM STATUS
CREDITS: 0 / 3
source.yaml(1L)
AI Audit Console
> Waiting for pre-flight audit...
AirLock HYBRID KERNEL: Active
TECHNICAL WHITEPAPER & ARCHITECTURE SPECUpdated 2026

Fixing SealedSecrets 'controller failed to unseal': Private Key Mismatch Diagnosis and Recovery

Threat/Impact Level: CRITICAL | Exploitability/Downtime Risk: HIGH | Time to Fix: 15–45 mins depending on key backup availability

TL;DR

  • What broke: The SealedSecrets controller's active private key does not match the key that was used to encrypt the SealedSecret resource — decryption fails hard, secret is never mounted, pod stays in CreateContainerConfigError or Pending.
  • How to fix it: Identify the correct key fingerprint, restore the original private key secret into kube-system, restart the controller, or re-seal all affected secrets against the current cluster public key.
  • Shortcut: Use our Client-Side Sandbox above to paste your SealedSecret YAML and controller logs — it will auto-identify the fingerprint mismatch and generate the corrected kubeseal re-encryption commands locally in your browser.

The Incident (What Does the Error Mean?)

Raw controller log output:

E0612 03:14:22.901847       1 controller.go:166] failed to unseal secret default/db-credentials: no key could decrypt secret
E0612 03:14:22.901901       1 controller.go:166] failed to unseal secret production/api-keys: no key could decrypt secret

Or the variant with explicit fingerprint rejection:

error unsealing secret: crypto/rsa: decryption error
no private key matched fingerprint: a3:f1:9c:44:...

Immediate consequence: Every SealedSecret object encrypted against the old key becomes permanently unreadable by the current controller. Kubernetes Secret objects are never created. Every workload referencing those secrets — Deployments, StatefulSets, CronJobs — fails to start. In production, this is a full application outage.


The Attack Vector / Blast Radius

This failure mode has two distinct causes, both catastrophic:

Cause 1 — Cluster rebuild / DR failover without key backup. The SealedSecrets controller auto-generates a fresh RSA-4096 key pair on first boot. If you rebuilt the cluster (disaster recovery, migration, cluster upgrade gone wrong) without exporting and re-importing the original sealed-secrets-key secret from kube-system, every single previously-sealed secret in your GitOps repo is now permanently unrecoverable without the plaintext originals. Your entire secret store is bricked.

Cause 2 — Accidental key rotation or controller reinstall. Helm upgrade --force, deleting the controller pod's persistent key secret, or reinstalling via a Helm chart that doesn't preserve existing keys generates a new key. Same result.

Blast radius:

  • All namespaces using SealedSecrets lose secret injection simultaneously.
  • CI/CD pipelines that apply SealedSecret manifests from Git will appear to succeed (kubectl apply returns OK) but the controller silently fails to decrypt — no alerting by default.
  • If you store DB credentials, API keys, or TLS certs as SealedSecrets, every downstream service is down.
  • If the original private key is gone and you have no plaintext backup, the secrets are cryptographically unrecoverable. You must rotate all credentials from scratch.

How to Fix It

Step 0 — Confirm the mismatch

# Get the fingerprint the SealedSecret was encrypted against
kubectl get sealedsecret db-credentials -n default -o jsonpath='{.spec.encryptedData}' | \
  kubeseal --validate --controller-namespace kube-system 2>&1

# List all private keys currently known to the controller
kubectl get secret -n kube-system -l sealedsecrets.bitnami.com/sealed-secrets-key

If the fingerprint in the error does not appear in the controller's key list, the key is missing.


Basic Fix — Restore the original private key from backup

If you exported the key before the incident (you should have — see Prevention below):

# Re-import the backed-up key into kube-system
kubectl apply -f sealed-secrets-master-key-backup.yaml -n kube-system

# Restart controller to pick up the restored key
kubectl rollout restart deployment/sealed-secrets-controller -n kube-system

# Verify controller now lists the restored key
kubectl logs -n kube-system deployment/sealed-secrets-controller | grep -i "key"

Enterprise Best Practice — Re-seal all secrets against the new cluster key

If the original key is gone, you must re-seal from plaintext. This requires you to have the original secret values (from a vault, HSM, or password manager — not from the cluster).

- # Old workflow: seal once, commit, forget the plaintext
- kubeseal --format yaml < secret-plain.yaml > sealed-secret.yaml
- # Result: sealed against a key that no longer exists

+ # Correct enterprise workflow: always seal against the CURRENT cluster's public cert
+ # Step 1: Export the active public cert explicitly
+ kubeseal --fetch-cert \
+   --controller-namespace kube-system \
+   --controller-name sealed-secrets-controller \
+   > current-cluster-pub-cert.pem
+
+ # Step 2: Re-seal using the fetched cert (works offline, no cluster API dependency)
+ kubeseal --format yaml \
+   --cert current-cluster-pub-cert.pem \
+   < secret-plain.yaml > sealed-secret-new.yaml
+
+ # Step 3: Commit sealed-secret-new.yaml to Git, delete the old one
+ # Step 4: Store current-cluster-pub-cert.pem in your secrets manager for DR

For scope-hardening, always use --scope namespace-wide or --scope strict (default) to prevent cross-namespace secret theft:

- kubeseal --format yaml < secret.yaml > sealed.yaml
+ kubeseal --format yaml --scope strict < secret.yaml > sealed.yaml
+ # 'strict' binds the sealed secret to BOTH the name AND namespace
+ # An attacker who can write to another namespace cannot replay this sealed secret

💡 Tired of pasting proprietary configs into ChatGPT? Generic AI tools log your company's ARNs, DB strings, and private keys. StackEngine is a zero-backend, pure browser-native utility. Drop your failing config into the sandbox above. We redact your secrets locally in the browser and auto-generate the refactored code using your own API key.


Prevention in CI/CD

1. Backup the master key on every cluster provisioning run — non-negotiable.

# Export ALL sealed-secrets keys (run this immediately after cluster bootstrap)
kubectl get secret -n kube-system \
  -l sealedsecrets.bitnami.com/sealed-secrets-key \
  -o yaml > sealed-secrets-master-key-backup.yaml

# Encrypt and store in your secrets manager (Vault, AWS Secrets Manager, etc.)
vault kv put secret/k8s/sealed-secrets-master-key \
  [email protected]

2. Add a CI validation step that detects key mismatch before merge:

# .github/workflows/validate-sealed-secrets.yaml
- name: Validate SealedSecrets can be unsealed
  run: |
    kubeseal --validate \
      --controller-namespace kube-system \
      < ${{ matrix.sealed_secret_file }}

This step will fail the PR if the SealedSecret was encrypted against a key the current cluster doesn't hold.

3. Use Checkov policy CKV_K8S_35 and add a custom OPA/Conftest rule:

# conftest policy: deny SealedSecrets without strict scope
deny[msg] {
  input.kind == "SealedSecret"
  not input.metadata.annotations["sealedsecrets.bitnami.com/cluster-wide"]
  not input.spec.template.metadata.namespace
  msg := sprintf("SealedSecret '%v' must declare an explicit namespace scope", [input.metadata.name])
}

4. Terraform/Helm: pin the controller version and use a pre-existing key secret:

- # Helm install with no key preservation — regenerates key on every reinstall
- helm upgrade --install sealed-secrets sealed-secrets/sealed-secrets \
-   -n kube-system

+ # Pre-create the key secret from backup BEFORE installing the controller
+ kubectl apply -f sealed-secrets-master-key-backup.yaml
+ helm upgrade --install sealed-secrets sealed-secrets/sealed-secrets \
+   -n kube-system \
+   --set keyrenewperiod=0  # Disable auto-rotation if using externally managed keys

5. Alert on unseal failures. The controller exposes a Prometheus metric sealed_secrets_controller_unseal_errors_total. Set a > 0 alert with severity: page.

Further Exploration / Matrix Routing