Kubectl Multi-Container Exec Auditor

The Incident (What Does the Error Mean?)

You ran something like:

kubectl exec -it my-pod -- /bin/sh

And the API server returned:

error: unable to upgrade connection: container not found ("")

or

error: unable to upgrade connection: container not found ("app")

Immediate consequence: You cannot exec into the pod. If this is a production incident and you need to inspect a running process, dump env vars, or run a diagnostic binary, you are blocked. Every minute counts.

The Kubernetes API server's /exec subresource requires an unambiguous container target. When a pod has two or more containers (sidecars, log shippers, service mesh proxies like Envoy, init containers, etc.), the API will not guess. If you omit -c or supply a name that doesn't match spec.containers[*].name exactly (case-sensitive), the request is rejected at the kubelet layer after the WebSocket upgrade attempt — hence the misleading "unable to upgrade connection" phrasing, which makes engineers chase network issues instead of the real cause.

The Attack Vector / Blast Radius

This is primarily an operational availability failure, but it has a secondary security surface:

Operational blast radius:

• During a live incident, engineers waste 10–20 minutes chasing kubelet logs, CNI issues, and RBAC policies before realizing the container name is wrong or missing.
• Automated runbooks and break-glass scripts that hardcode a single container name silently break the moment a sidecar (Istio, Datadog agent, Vault injector) is injected into the pod by a mutating webhook — without any change to your script. The injection happens transparently at admission time.
• CI/CD smoke tests using kubectl exec for health validation fail with cryptic errors, blocking deployments.

Security surface:

• If your RBAC pods/exec verb is scoped to a role without container-level granularity, an engineer trying to exec into the app container might accidentally target a privileged sidecar (e.g., a Vault agent with mounted secrets or a Falco sensor) if they guess the wrong name. Least-privilege exec should be paired with correct targeting.
• Misconfigured admission webhooks that inject containers with names colliding with your app container name will cause this error and can mask injection failures.

Prevention in CI/CD

1. OPA/Gatekeeper — Enforce Explicit Container Names in Exec

Write a Gatekeeper ConstraintTemplate that rejects pods/exec admission requests where the containerName field is empty. This forces all exec calls — including those from CI pipelines — to be explicit.

2. Checkov / Static Analysis on Runbook Scripts

Add a shell linter step in CI that scans for kubectl exec invocations missing -c:

# In your CI pipeline (GitHub Actions, GitLab CI, etc.)
grep -rn 'kubectl exec' ./scripts/ | grep -v '\-c ' && \
  echo "ERROR: kubectl exec found without -c flag" && exit 1

3. Kyverno Policy — Admission-Time Container Name Validation

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-exec-container-name
spec:
  rules:
  - name: check-exec-container
    match:
      resources:
        kinds: ["PodExecOptions"]
    validate:
      message: "kubectl exec must specify a container name explicitly via -c."
      pattern:
        container: "?*"

4. Mutating Webhook Awareness

Document which webhooks inject sidecars into your namespaces. After any Istio/Linkerd/Vault injection policy change, run a pod audit:

kubectl get pods -n production -o json | \
  jq '.items[] | {name: .metadata.name, containers: [.spec.containers[].name]}'

Pipe this into your CMDB or alert if container count deviates from the expected baseline.