How to Fix Nginx proxy_ssl_verify Failed 'Self Signed Certificate' for Upstream

Threat/Impact Level: HIGH | Exploitability/Downtime Risk: HIGH | Time to Fix: 10 mins

---

TL;DR

• What broke: Nginx has proxy_ssl_verify on but no proxy_ssl_trusted_certificate pointing to the upstream's CA bundle, so TLS handshake to the upstream fails hard with SSL_do_handshake() failed.
• How to fix it: Either supply the self-signed cert (or its CA) via proxy_ssl_trusted_certificate, or — in controlled internal networks — scope the trust correctly. Do not just flip proxy_ssl_verify off; that silently removes all upstream TLS integrity.
• Fast path: Use our Client-Side Sandbox below to auto-refactor this — paste your location block and get corrected directives without sending your config to a third-party server.

---

The Incident (What Does the Error Mean?)

Raw error from nginx/error.log:

2024/06/10 03:17:42 [error] 31#31: *1042 SSL_do_handshake() failed 
(SSL: error:1416F086:SSL routines:tls_process_server_certificate:
certificate verify failed) while SSL handshaking to upstream, 
client: 10.0.1.55, server: api.internal, request: "POST /v2/ingest HTTP/1.1",
upstream: "https://10.0.2.88:8443/v2/ingest", host: "api.internal"

Immediate consequence: Every proxied request to that upstream returns a 502 Bad Gateway. If this upstream is your auth service, payment processor, or internal API gateway, the blast is total — all dependent services fail simultaneously. Nginx does not retry on TLS handshake failures by default.

The root mechanics: proxy_ssl_verify on instructs Nginx's OpenSSL context to validate the upstream certificate chain against a trusted CA store. When the upstream presents a self-signed cert, there is no chain to a known root CA. OpenSSL returns X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT or X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN. Nginx maps this to a 502 and logs the SSL handshake failure.

---

The Attack Vector / Blast Radius

This misconfiguration creates a binary failure state with a dangerous temptation: engineers under outage pressure will set proxy_ssl_verify off as the "quick fix." That is the actual security vulnerability.

With proxy_ssl_verify off:

• Nginx will connect to any host on that upstream IP/port regardless of certificate identity.
• An attacker with network access (compromised pod, SSRF, ARP spoofing on a flat internal network) can intercept the connection and present any certificate. Nginx accepts it.
• All data — including auth tokens, session cookies, internal API keys — transits to the attacker's endpoint.
• In Kubernetes environments with CNI misconfigurations or in AWS VPCs with overly permissive security groups, lateral movement to the upstream address is trivially achievable.

Blast radius if proxy_ssl_verify off is deployed to production:

1. Full upstream MITM exposure on every proxied request.
2. No alerting — Nginx logs show 200 OK for successfully proxied (intercepted) requests.
3. Compliance violations: PCI-DSS Req 6.5.4, SOC 2 CC6.7, and HIPAA §164.312(e)(2)(i) all require encryption integrity in transit.

---

How to Fix It (The Solution)

Basic Fix — Trust the Specific Self-Signed Certificate

Export the upstream's self-signed certificate to a PEM file on the Nginx host:

openssl s_client -connect 10.0.2.88:8443 -showcerts </dev/null 2>/dev/null \
  | openssl x509 -outform PEM > /etc/nginx/certs/upstream-selfsigned.pem

Then reference it in your Nginx config:

 location /v2/ {
     proxy_pass                  https://10.0.2.88:8443;
-    proxy_ssl_verify            off;
+    proxy_ssl_verify            on;
+    proxy_ssl_trusted_certificate /etc/nginx/certs/upstream-selfsigned.pem;
+    proxy_ssl_verify_depth      2;
+    proxy_ssl_server_name       on;
+    proxy_ssl_name              upstream.internal.svc;
     proxy_set_header Host       $host;
     proxy_set_header X-Real-IP  $remote_addr;
 }

proxy_ssl_server_name on + proxy_ssl_name forces Nginx to send the correct SNI in the ClientHello, which is required when the upstream cert's CN/SAN must match the presented hostname rather than the raw IP.

---

Enterprise Best Practice — Internal PKI / Intermediate CA

Self-signed certs per-service are an operational nightmare. The correct architecture is an internal CA (HashiCorp Vault PKI, AWS Private CA, or cfssl) issuing short-lived certs to all upstream services. Nginx trusts the internal CA root only.

 # nginx.conf — http context
+ssl_trusted_certificate     /etc/nginx/certs/internal-root-ca.pem;
+ssl_verify_depth            3;

 # location block
 location /v2/ {
     proxy_pass                    https://upstream-service:8443;
-    proxy_ssl_verify              off;
+    proxy_ssl_verify              on;
+    proxy_ssl_trusted_certificate /etc/nginx/certs/internal-root-ca.pem;
+    proxy_ssl_verify_depth        3;
+    proxy_ssl_server_name         on;
     # mTLS — present Nginx's own client cert to upstream
+    proxy_ssl_certificate         /etc/nginx/certs/nginx-client.crt;
+    proxy_ssl_certificate_key     /etc/nginx/certs/nginx-client.key;
     proxy_set_header Host         $host;
 }

With Vault PKI, certs rotate automatically. Nginx picks up new certs on reload (nginx -s reload or via inotify-based reload sidecars in Kubernetes). No manual cert exports. No stale self-signed certs expiring at 3 AM.

---

💡 Tired of pasting proprietary configs into ChatGPT? Generic AI tools log your company's ARNs, DB strings, and private keys. StackEngine is a zero-backend, pure Client-Side WASM utility. Drop your failing config into the sandbox above. We redact your secrets locally in the browser and auto-generate the refactored code using your own API key.

---

Prevention in CI/CD

1. Checkov — block proxy_ssl_verify off at PR time:

Checkov doesn't have a native Nginx check for this, so write a custom policy:

# checkov/custom/nginx_ssl_verify.py
from checkov.common.models.enums import CheckResult, CheckCategories
from checkov.nginx.checks.base_nginx_check import BaseNginxCheck

class NginxProxySslVerifyOff(BaseNginxCheck):
    def __init__(self):
        name = "Ensure proxy_ssl_verify is not disabled"
        id = "CKV_NGINX_CUSTOM_01"
        super().__init__(name=name, check_id=id)

    def check_resource_config(self, config):
        if 'proxy_ssl_verify' in config and config['proxy_ssl_verify'] == ['off']:
            return CheckResult.FAILED, config
        return CheckResult.PASSED, config

2. OPA/Conftest — enforce trusted certificate directive presence:

# policy/nginx_ssl.rego
package nginx.ssl

deny[msg] {
  input.proxy_ssl_verify == "on"
  not input.proxy_ssl_trusted_certificate
  msg := "proxy_ssl_verify is on but proxy_ssl_trusted_certificate is not set"
}

deny[msg] {
  input.proxy_ssl_verify == "off"
  msg := "proxy_ssl_verify must not be disabled; configure a trusted CA bundle"
}

Run in CI:

conftest test nginx.conf --policy policy/nginx_ssl.rego

3. Certificate expiry monitoring:

Self-signed certs expire silently. Add a Prometheus blackbox exporter probe on all upstream TLS endpoints with a ssl_expiry_days < 30 alert. When the cert expires, proxy_ssl_verify on will trigger the same 502 outage.

# blackbox.yml
modules:
  https_upstream_check:
    prober: tcp
    tcp:
      tls: true
      tls_config:
        ca_file: /etc/prometheus/certs/internal-root-ca.pem

4. Rotate away from self-signed certs entirely. Every self-signed cert in your infrastructure is a future 3 AM incident. Internal PKI with 90-day cert TTLs and automated rotation is the only durable solution.