How to Fix FluxCD 'reconcile failed: GitRepository not ready' – Repo Unreachable Debugging Guide

Threat/Impact Level: HIGH | Downtime Risk: HIGH | Time to Fix: 5–20 mins

---

TL;DR

• What broke: FluxCD's source-controller cannot reach the Git remote — the GitRepository object is stuck in False/NotReady, which cascades and blocks every Kustomization and HelmRelease that depends on it.
• How to fix it: Validate the URL scheme, confirm the secretRef credential secret exists and is correctly typed, check egress NetworkPolicies and corporate proxy settings, and verify TLS/CA trust.
• Fast path: Use our Client-Side Sandbox above to paste your GitRepository manifest — it auto-diagnoses and refactors the YAML without sending your tokens anywhere.

---

The Incident (What Does the Error Mean?)

Raw output from flux get sources git -A:

NAMESPACE    NAME          READY   MESSAGE                                                              REVISION   SUSPENDED
flux-system  app-repo      False   failed to checkout and determine revision: unable to clone           
                                   'https://github.com/org/private-repo': Get "https://github.com":    
                                   dial tcp: i/o timeout

Or via kubectl describe gitrepository app-repo -n flux-system:

Status:
  Conditions:
    Message:   failed to checkout and determine revision: 
               unable to clone '[email protected]:org/repo.git': 
               ssh: handshake failed: knownhosts: key mismatch
    Reason:    GitOperationFailed
    Status:    False
    Type:      Ready

Immediate consequence: Every Kustomization referencing this source enters dependency not ready state. No manifests are applied. Active workloads are not reconciled. A config drift or rollback becomes invisible to Flux until this is resolved.

---

The Attack Vector / Blast Radius

This is not a cosmetic warning. The blast radius is cluster-wide for every resource in the dependency chain:

1. All Kustomizations referencing this source stop reconciling. If a security patch or RBAC change was pushed to Git, it will never land.
2. HelmReleases using sourceRef to this GitRepository are frozen. A broken image tag or misconfigured value stays deployed indefinitely.
3. flux reconcile commands will loop and timeout, consuming operator time during an incident.
4. SSH key mismatch errors are particularly dangerous — they can indicate a MITM condition on the Git remote's host key (e.g., GitHub rotating keys, or a corporate proxy intercepting TLS). Do not blindly --insecure-skip-tls-verify your way out of this.
5. Expired deploy tokens silently break reconciliation with a 401 that looks identical to a network timeout in some controller versions.

---

How to Fix It

Step 1: Triage the Exact Failure Reason

# Get the precise status message
kubectl get gitrepository app-repo -n flux-system -o jsonpath='{.status.conditions[*].message}'

# Check source-controller logs for the raw error
kubectl logs -n flux-system deploy/source-controller --since=10m | grep -E 'error|failed|unable'

# Force an immediate reconcile attempt
flux reconcile source git app-repo -n flux-system --timeout=60s

---

Fix A: Wrong URL or Protocol Mismatch

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: app-repo
  namespace: flux-system
spec:
- url: [email protected]:org/repo  # missing .git suffix, causes clone failure
+ url: ssh://[email protected]/org/repo.git
  interval: 1m0s
  secretRef:
    name: github-ssh-key

Note: For HTTPS, use https://github.com/org/repo. For SSH, Flux requires the ssh:// scheme explicitly — the SCP-style [email protected]:org/repo.git is not universally accepted across controller versions.

---

Fix B: Missing or Malformed SSH Secret

The secretRef must contain identity, identity.pub, and known_hosts keys. A missing known_hosts causes the knownhosts: key mismatch error.

# Wrong secret structure (missing known_hosts)
apiVersion: v1
kind: Secret
metadata:
  name: github-ssh-key
  namespace: flux-system
type: Opaque
data:
- identity: <base64-private-key>
- identity.pub: <base64-public-key>
# known_hosts is absent — source-controller rejects the host

+ identity: <base64-private-key>
+ identity.pub: <base64-public-key>
+ known_hosts: <base64-of-github-known-hosts-entry>

Generate the correct secret:

# Scan and capture GitHub's current host key
ssh-keyscan github.com > /tmp/known_hosts 2>/dev/null

# Create the Flux SSH secret correctly
flux create secret git github-ssh-key \
  --url=ssh://[email protected]/org/repo.git \
  --private-key-file=./deploy_key \
  --known-hosts-file=/tmp/known_hosts \
  --namespace=flux-system

---

Fix C: Egress Blocked by NetworkPolicy

If your cluster has default-deny egress, source-controller cannot reach the Git remote.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-source-controller-egress
  namespace: flux-system
spec:
  podSelector:
    matchLabels:
      app: source-controller
  policyTypes:
  - Egress
  egress:
- [] # no egress rules — all outbound blocked
+ - ports:
+   - port: 443
+     protocol: TCP
+   - port: 22
+     protocol: TCP

---

Fix D: Corporate Proxy / Self-Signed CA

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: app-repo
  namespace: flux-system
spec:
  url: https://git.internal.corp/org/repo.git
  interval: 1m0s
- # No CA reference — TLS verification fails against internal CA
+ certSecretRef:
+   name: internal-ca-cert  # Secret must contain 'ca.crt' key
  secretRef:
    name: gitlab-https-token

---

Enterprise Best Practice: Automate Secret Rotation with External Secrets Operator

Hardcoded deploy tokens expire. Wire ExternalSecret to Vault or AWS Secrets Manager so the SSH key secret auto-rotates:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: github-ssh-key
  namespace: flux-system
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: github-ssh-key
+   template:
+     type: Opaque
  data:
+   - secretKey: identity
+     remoteRef:
+       key: secret/flux/github-deploy-key
+       property: private_key
+   - secretKey: known_hosts
+     remoteRef:
+       key: secret/flux/github-deploy-key
+       property: known_hosts

---

💡 Tired of pasting proprietary configs into ChatGPT? Generic AI tools log your company's ARNs, DB strings, and private keys. StackEngine is a zero-backend, pure Client-Side WASM utility. Drop your failing GitRepository manifest into the sandbox above. We redact your secrets locally in the browser and auto-generate the refactored code using your own API key.

---

Prevention in CI/CD

1. Validate GitRepository Manifests Pre-Merge with Kubeconform

# In your GitHub Actions / GitLab CI pipeline
kubeconform -strict -schema-location default \
  -schema-location 'https://raw.githubusercontent.com/fluxcd/flux2/main/config/crd/bases/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' \
  ./clusters/production/flux-system/

2. OPA/Gatekeeper Policy: Enforce secretRef on Private Repos

package fluxcd.gitrepository

deny[msg] {
  input.kind == "GitRepository"
  not input.spec.secretRef
  msg := sprintf("GitRepository '%v' has no secretRef — will fail on private repos", [input.metadata.name])
}

3. Flux Alerts to Slack/PagerDuty on NotReady

apiVersion: notification.toolkit.fluxcd.io/v1beta3
kind: Alert
metadata:
  name: gitrepository-not-ready
  namespace: flux-system
spec:
  summary: "GitRepository failed to reconcile"
  providerRef:
    name: slack-ops
  eventSeverity: error
  eventSources:
    - kind: GitRepository
      name: "*"
  inclusionList:
    - ".*failed.*"
    - ".*NotReady.*"

4. Checkov Scan in CI

checkov -d ./clusters/ --framework kubernetes \
  --check CKV_K8S_35  # Secrets not hardcoded as env vars

5. Smoke-Test Connectivity Before Cluster Bootstrap

# Run from a pod in the flux-system namespace to validate reachability
kubectl run ssh-test --rm -it --image=alpine -n flux-system -- \
  sh -c "apk add openssh-client && ssh -T [email protected] -o StrictHostKeyChecking=yes"