Harbor Image Pull Secret Pod Spec Checker

The Incident (What Does the Error Mean?)

You'll see one or both of these in kubectl describe pod <pod-name>:

Failed to pull image "harbor.internal.corp/project/app:1.4.2": 
  rpc error: code = Unknown desc = failed to pull and unpack image 
  "harbor.internal.corp/project/app:1.4.2": failed to resolve reference 
  "harbor.internal.corp/project/app:1.4.2": unexpected status code 401 Unauthorized

Warning  Failed     3m    kubelet  Error: ErrImagePull
Warning  Failed     2m    kubelet  Error: ImagePullBackOff

Kubernetes attempted to pull from a private Harbor registry without presenting credentials. Harbor returned 401. The kubelet enters exponential backoff. The pod never starts. Any Deployment rollout is dead in the water.

The Attack Vector / Blast Radius

This isn't just an ops inconvenience — the absence of a properly scoped pull secret is a credential hygiene failure with real blast radius:

• Deployment outage: Every pod replica fails simultaneously. If this hits during a rollout, your previous ReplicaSet may also be scaling down, leaving you with zero healthy pods.
• Namespace sprawl risk: Teams often "fix" this by making the Harbor project public. A public Harbor project exposes internal images to anyone with network access — including your base images that may contain internal CA certs, hardcoded staging endpoints, or proprietary binaries.
• Secret scope creep: The other common "fix" is attaching the pull secret to the default ServiceAccount, which then grants every pod in the namespace implicit pull access — violating least-privilege.
• Stale credential time bomb: If the secret was created once and the Harbor robot account password rotated, every namespace referencing that secret silently breaks at the next image pull — often discovered only during a node drain or pod eviction.

Prevention in CI/CD

OPA / Gatekeeper Policy — Block Deployments Without imagePullSecrets

package kubernetes.admission

deny[msg] {
  input.request.kind.kind == "Pod"
  container := input.request.object.spec.containers[_]
  startswith(container.image, "harbor.internal.corp/")
  count(input.request.object.spec.imagePullSecrets) == 0
  msg := sprintf("Pod '%v' pulls from Harbor but has no imagePullSecrets defined.", 
    [input.request.object.metadata.name])
}

Checkov — Static Analysis in Pipeline

# .checkov.yaml
check:
  - CKV_K8S_27  # Do not admit containers that wish to share the host IPC namespace
# Add custom check for imagePullSecrets via checkov custom policies

Run in CI:

checkov -d ./k8s-manifests --framework kubernetes --check CKV_K8S_27

Helm — Enforce via values.schema.json

{
  "properties": {
    "imagePullSecrets": {
      "type": "array",
      "minItems": 1,
      "description": "At least one imagePullSecret is required for Harbor registry access."
    }
  },
  "required": ["imagePullSecrets"]
}

Harbor Robot Account Rotation — Automate It

• Use External Secrets Operator with your vault (Vault, AWS Secrets Manager) to auto-rotate the harbor-pull-secret before expiry.
• Set Harbor robot account expiry to 90 days max. Configure ESO to sync 7 days before expiry.
• Never use a human Harbor account as a pull credential — robot accounts are auditable, revocable, and project-scoped.

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: harbor-pull-secret-sync
  namespace: your-app-namespace
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: harbor-pull-secret
    template:
      type: kubernetes.io/dockerconfigjson
  data:
    - secretKey: .dockerconfigjson
      remoteRef:
        key: secret/harbor/robot-ci-puller
        property: dockerconfigjson