Why does Let's Encrypt return 404 for the ACME challenge even though my site loads fine over HTTPS?

Your HTTPS site works because the SSL cert (however it was obtained) is already loaded. The ACME HTTP-01 challenge operates exclusively over plain HTTP on port 80. If your Nginx config redirects all port 80 traffic to HTTPS before serving the challenge file — which is the most common production setup — Let's Encrypt's validation server never reaches the token file. The site appearing healthy over HTTPS masks the broken renewal path entirely.

Can I use DNS-01 challenge instead to avoid this Nginx configuration problem entirely?

Yes. DNS-01 challenge (`certbot certonly --dns- `) validates domain ownership by placing a TXT record in your DNS zone instead of serving a file over HTTP. This bypasses Nginx completely and works even when port 80 is firewalled. The tradeoff: you need a supported DNS provider plugin (Cloudflare, Route53, etc.) and DNS propagation adds 30–120 seconds to the issuance time. For wildcard certificates (`*.yourdomain.com`), DNS-01 is the only supported method — HTTP-01 cannot issue wildcards.

Certbot says the challenge succeeded but Nginx still returns 404 in my browser — what's happening?

If Certbot reports success but you manually see a 404 browsing to `/.well-known/acme-challenge/`, that's expected post-validation behavior. Certbot cleans up the token file immediately after the CA confirms the challenge. The directory will be empty. The real test is running `certbot renew --dry-run` — if that exits 0, your configuration is correct. If it fails with 404, your Nginx config is still broken regardless of what a manual browser test shows.

How to Fix Let's Encrypt ACME Challenge 404 Not Found in Nginx (.well-known/acme-challenge)

Threat/Impact Level: CRITICAL | Exploitability/Downtime Risk: HIGH | Time to Fix: 10 mins

TL;DR

What broke: Nginx returned 404 Not Found when Let's Encrypt's ACME server tried to validate http://yourdomain.com/.well-known/acme-challenge/<token> — certificate issuance failed completely.
How to fix it: Add an explicit location ^~ /.well-known/acme-challenge/ block that serves the webroot directory before any HTTPS redirect or catch-all rule fires.
Use our Client-Side Sandbox above to paste your failing Nginx server block and auto-generate the corrected config with the ACME location block injected in the right order.

The Incident (What Does the Error Mean?)

During certbot --nginx or certbot certonly --webroot, the ACME CA (Let's Encrypt) makes an outbound HTTP GET to:

GET http://yourdomain.com/.well-known/acme-challenge/<random_token>

Your server responded:

HTTP/1.1 404 Not Found

Certbot output typically looks like this:

Detail: 404 :: urn:ietf:params:acme:error:unauthorized ::
Invalid response from http://yourdomain.com/.well-known/acme-challenge/Xk9mQ...:
"<html>\r\n<head><title>404 Not Found</title></head>"

Immediate consequence: Certificate issuance is aborted. If this is a renewal, your existing cert will expire. If this is a new deployment, TLS never comes up. Your site runs on HTTP or throws an SSL handshake error.

The Attack Vector / Blast Radius

This isn't a hacker exploit — it's a self-inflicted outage vector with a hard deadline. Let's Encrypt certs expire in 90 days. Auto-renewal runs at ~60 days. If your Nginx config has been broken since the last successful issuance, you will hit a hard expiry with zero warning until browsers start throwing NET::ERR_CERT_DATE_INVALID to all users.

Cascading failure chain:

Certbot renewal cron/systemd timer fires silently.
ACME challenge returns 404 — renewal fails.
No alert fires because the cert hasn't expired yet.
30 days later: cert expires. All HTTPS traffic breaks. Load balancers start dropping connections. API clients get TLS errors.
If you're running return 301 https://$host$request_uri; on port 80 without an ACME exception, every renewal attempt will be permanently broken — you cannot even get a new cert to recover.

The most dangerous scenario: a port 80 → HTTPS redirect that was added after initial cert issuance. The cert was obtained once manually, the redirect was added, and nobody noticed renewals were silently failing for months.

How to Fix It

Root Cause Checklist

Before touching config, identify which failure mode you have:

Symptom	Cause
Port 80 returns 301 to HTTPS for all paths	HTTPS redirect has no ACME exception
404 on the token path specifically	Wrong or missing `root` in webroot location
Connection refused on port 80	Nginx not listening on 80 at all
403 Forbidden	Filesystem permissions on `.well-known/` dir

Basic Fix — Add the ACME Location Block Before Your Redirect

 server {
     listen 80;
     server_name yourdomain.com www.yourdomain.com;
 
-    # This catch-all redirect kills ACME challenges
-    return 301 https://$host$request_uri;
+    # ACME challenge MUST be served over plain HTTP — no redirect
+    location ^~ /.well-known/acme-challenge/ {
+        root /var/www/certbot;
+        default_type "text/plain";
+        allow all;
+    }
+
+    # Redirect everything else to HTTPS
+    location / {
+        return 301 https://$host$request_uri;
+    }
 }

Then create the webroot directory and set permissions:

mkdir -p /var/www/certbot/.well-known/acme-challenge
chown -R www-data:www-data /var/www/certbot
nginx -t && systemctl reload nginx

Re-run certbot pointing to the same webroot:

certbot certonly --webroot -w /var/www/certbot -d yourdomain.com -d www.yourdomain.com

Enterprise Best Practice — Dedicated Certbot Webroot with Shared Include

For multi-vhost environments, centralize the ACME config as a shared include so every server block gets it automatically and you never miss it again.

/etc/nginx/snippets/acme-challenge.conf:

location ^~ /.well-known/acme-challenge/ {
    root /var/www/certbot;
    default_type "text/plain";
    try_files $uri =404;
    allow all;
}

Per-vhost server block:

 server {
     listen 80;
     server_name yourdomain.com;
 
+    include snippets/acme-challenge.conf;
 
     location / {
         return 301 https://$host$request_uri;
     }
 }

For Docker/containerized Nginx + Certbot (the other common failure mode where the webroot volume isn't mounted):

 # docker-compose.yml
 services:
   nginx:
     volumes:
-      - ./nginx.conf:/etc/nginx/nginx.conf
+      - ./nginx.conf:/etc/nginx/nginx.conf
+      - certbot-webroot:/var/www/certbot   # MUST be shared with certbot container
   certbot:
     image: certbot/certbot
     volumes:
+      - certbot-webroot:/var/www/certbot
+      - ./letsencrypt:/etc/letsencrypt
+volumes:
+  certbot-webroot:

💡 Tired of pasting proprietary configs into ChatGPT? Generic AI tools log your company's ARNs, DB strings, and private keys. StackEngine is a zero-backend, pure Client-Side WASM utility. Drop your failing config into the sandbox above. We redact your secrets locally in the browser and auto-generate the refactored code using your own API key.

Prevention in CI/CD

1. Validate Nginx config in your pipeline before deploy:

# In your CI step, after generating nginx.conf from templates
nginx -t -c /path/to/generated/nginx.conf

# Grep-based sanity check — fail the pipeline if ACME block is missing
grep -q '.well-known/acme-challenge' /etc/nginx/sites-enabled/* || \
  (echo "FATAL: ACME challenge location block missing" && exit 1)

2. Monitor cert expiry with Prometheus/Alertmanager:

# alertmanager rule — fire 21 days before expiry
- alert: SSLCertExpiringWithin21Days
  expr: probe_ssl_earliest_cert_expiry - time() < 86400 * 21
  labels:
    severity: critical
  annotations:
    summary: "TLS cert for {{ $labels.instance }} expires in < 21 days"

3. Use checkov or conftest with OPA to enforce the ACME block policy in IaC:

# opa policy: nginx_acme_policy.rego
package nginx

deny[msg] {
    input.server[_].listen == "80"
    not has_acme_location(input.server[_])
    msg := "Port 80 server block is missing /.well-known/acme-challenge/ location"
}

has_acme_location(server) {
    server.location[_].path == "^~ /.well-known/acme-challenge/"
}

4. Enable Certbot systemd timer with failure alerting:

systemctl enable --now certbot.timer

# Override to send email on failure
systemctl edit certbot.service
# Add:
# [Service]
# ExecStartPost=/usr/local/bin/notify-on-failure.sh

5. Test renewal in dry-run mode weekly from CI:

certbot renew --dry-run --quiet || \
  curl -X POST https://hooks.slack.com/... -d '{"text":"Certbot dry-run FAILED on prod"}'