Nginx Proxy SSL Verify Self-Signed Resolver

The Incident (What Does the Error Mean?)

Raw error from nginx/error.log:

2024/06/10 03:17:42 [error] 31#31: *1042 SSL_do_handshake() failed 
(SSL: error:1416F086:SSL routines:tls_process_server_certificate:
certificate verify failed) while SSL handshaking to upstream, 
client: 10.0.1.55, server: api.internal, request: "POST /v2/ingest HTTP/1.1",
upstream: "https://10.0.2.88:8443/v2/ingest", host: "api.internal"

Immediate consequence: Every proxied request to that upstream returns a 502 Bad Gateway. If this upstream is your auth service, payment processor, or internal API gateway, the blast is total — all dependent services fail simultaneously. Nginx does not retry on TLS handshake failures by default.

The root mechanics: proxy_ssl_verify on instructs Nginx's OpenSSL context to validate the upstream certificate chain against a trusted CA store. When the upstream presents a self-signed cert, there is no chain to a known root CA. OpenSSL returns X509_V_ERR_DEPTH_ZERO_SELF_SIGNED_CERT or X509_V_ERR_SELF_SIGNED_CERT_IN_CHAIN. Nginx maps this to a 502 and logs the SSL handshake failure.

The Attack Vector / Blast Radius

This misconfiguration creates a binary failure state with a dangerous temptation: engineers under outage pressure will set proxy_ssl_verify off as the "quick fix." That is the actual security vulnerability.

With proxy_ssl_verify off:

• Nginx will connect to any host on that upstream IP/port regardless of certificate identity.
• An attacker with network access (compromised pod, SSRF, ARP spoofing on a flat internal network) can intercept the connection and present any certificate. Nginx accepts it.
• All data — including auth tokens, session cookies, internal API keys — transits to the attacker's endpoint.
• In Kubernetes environments with CNI misconfigurations or in AWS VPCs with overly permissive security groups, lateral movement to the upstream address is trivially achievable.

Blast radius if proxy_ssl_verify off is deployed to production:

1. Full upstream MITM exposure on every proxied request.
2. No alerting — Nginx logs show 200 OK for successfully proxied (intercepted) requests.
3. Compliance violations: PCI-DSS Req 6.5.4, SOC 2 CC6.7, and HIPAA §164.312(e)(2)(i) all require encryption integrity in transit.

Prevention in CI/CD

1. Checkov — block proxy_ssl_verify off at PR time:

Checkov doesn't have a native Nginx check for this, so write a custom policy:

# checkov/custom/nginx_ssl_verify.py
from checkov.common.models.enums import CheckResult, CheckCategories
from checkov.nginx.checks.base_nginx_check import BaseNginxCheck

class NginxProxySslVerifyOff(BaseNginxCheck):
    def __init__(self):
        name = "Ensure proxy_ssl_verify is not disabled"
        id = "CKV_NGINX_CUSTOM_01"
        super().__init__(name=name, check_id=id)

    def check_resource_config(self, config):
        if 'proxy_ssl_verify' in config and config['proxy_ssl_verify'] == ['off']:
            return CheckResult.FAILED, config
        return CheckResult.PASSED, config

2. OPA/Conftest — enforce trusted certificate directive presence:

# policy/nginx_ssl.rego
package nginx.ssl

deny[msg] {
  input.proxy_ssl_verify == "on"
  not input.proxy_ssl_trusted_certificate
  msg := "proxy_ssl_verify is on but proxy_ssl_trusted_certificate is not set"
}

deny[msg] {
  input.proxy_ssl_verify == "off"
  msg := "proxy_ssl_verify must not be disabled; configure a trusted CA bundle"
}

Run in CI:

conftest test nginx.conf --policy policy/nginx_ssl.rego

3. Certificate expiry monitoring:

Self-signed certs expire silently. Add a Prometheus blackbox exporter probe on all upstream TLS endpoints with a ssl_expiry_days < 30 alert. When the cert expires, proxy_ssl_verify on will trigger the same 502 outage.

# blackbox.yml
modules:
  https_upstream_check:
    prober: tcp
    tcp:
      tls: true
      tls_config:
        ca_file: /etc/prometheus/certs/internal-root-ca.pem

4. Rotate away from self-signed certs entirely. Every self-signed cert in your infrastructure is a future 3 AM incident. Internal PKI with 90-day cert TTLs and automated rotation is the only durable solution.