How to Fix Nginx X-Forwarded-For IP Concatenation Errors (Spoofing & Rate Limit Bypass)
Threat/Impact Level: HIGH | Exploitability/Downtime Risk: HIGH | Time to Fix: 10 mins
TL;DR
- What broke: Nginx is either blindly trusting the full
X-Forwarded-Forchain or appending IPs incorrectly, letting any client inject a fake originating IP into the header. - How to fix it: Use
ngx_http_realip_modulewith explicit trusted proxy CIDRs, enablereal_ip_recursive on, and stop reading$http_x_forwarded_fordirectly in access control logic. - Use our Client-Side Sandbox below to auto-refactor this — paste your
nginx.confand get a corrected config without sending your internal IPs to a third-party AI.
The Incident (What does the error mean?)
The symptom surfaces in access logs or your upstream application receiving malformed or spoofed client IPs:
# Access log showing concatenated garbage or attacker-injected IP at position [0]
192.168.1.1, 10.0.0.5, 203.0.113.99 ← upstream app reads leftmost = attacker-controlled
# Or your rate limiter keying on $http_x_forwarded_for directly
limit_req_zone $http_x_forwarded_for zone=api:10m rate=10r/s;
# An attacker sends: X-Forwarded-For: 1.2.3.4
# Nginx appends real IP → "1.2.3.4, 10.10.0.2"
# Rate limiter sees "1.2.3.4, 10.10.0.2" as the key → never matches a previous request → bypass
Immediate consequence: Rate limiting is neutered. IP-based allow/deny rules are bypassed. Audit logs record attacker-supplied IPs. Any upstream service trusting request.headers['x-forwarded-for'].split(',')[0] is reading attacker-controlled data.
The Attack Vector / Blast Radius
This is a header injection / IP spoofing primitive. The attack chain:
- Attacker sends
X-Forwarded-For: 127.0.0.1(or any allowlisted IP) in their request. - Nginx, configured without
real_ip_recursiveor with overly broadset_real_ip_from 0.0.0.0/0, either trusts this value wholesale or appends the real IP to the right — leaving the spoofed IP at position[0]. - Upstream app does
xff.split(',')[0].strip()→ reads127.0.0.1→ grants admin access, bypasses geo-block, or escapes rate limiting. - If
limit_req_zoneorlimit_conn_zonekeys on$http_x_forwarded_for(the raw header), the key is the full unsanitized string. Each unique injected value is a new zone key. This is also a zone exhaustion / memory DoS vector — flood with unique XFF values, blow out the shared memory zone.
Blast radius: WAF bypass, rate limit bypass, admin IP allowlist bypass, poisoned audit trail, potential shared memory DoS on the Nginx worker.
How to Fix It (The Solution)
Basic Fix — Trust Only Your Load Balancer, Resolve Real IP
http {
- # WRONG: Keying rate limit on raw, unsanitized header
- limit_req_zone $http_x_forwarded_for zone=api:10m rate=10r/s;
+ # CORRECT: Key on $binary_remote_addr after real IP resolution
+ limit_req_zone $binary_remote_addr zone=api:10m rate=10r/s;
server {
listen 80;
- # WRONG: No real IP module config — $remote_addr is the LB, not the client
- # and $http_x_forwarded_for is fully attacker-controlled
+ # CORRECT: Declare your trusted upstream proxy CIDRs
+ set_real_ip_from 10.0.0.0/8; # internal LB subnet
+ set_real_ip_from 172.16.0.0/12; # VPC NAT range
+ real_ip_header X-Forwarded-For;
+ real_ip_recursive on; # strips ALL trusted IPs from the right,
+ # leaving the first untrusted IP as $remote_addr
location /api/ {
- # WRONG: Access control on raw header
- if ($http_x_forwarded_for ~* "^10\.0\.0\.") {
- allow all;
- }
+ # CORRECT: Access control on resolved $remote_addr
+ allow 10.0.0.0/8;
+ deny all;
limit_req zone=api burst=20 nodelay;
proxy_pass http://backend;
- # WRONG: Forwarding the raw header allows the chain to grow unbounded
- proxy_set_header X-Forwarded-For $http_x_forwarded_for;
+ # CORRECT: Rebuild the header with only the verified client IP
+ proxy_set_header X-Forwarded-For $remote_addr;
+ # If you need the full chain for upstream logging, use:
+ # proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+ # BUT only after real_ip_recursive has resolved $remote_addr correctly
}
}
}
Enterprise Best Practice — Multi-Layer Proxy Topology (CDN → ALB → Nginx)
When you have a CDN (Cloudflare, CloudFront) in front of an ALB in front of Nginx, the XFF chain is 3+ hops. You need to declare all intermediate proxy CIDRs:
http {
+ # Layer 1: CDN egress IPs (fetch from provider API, automate this)
+ set_real_ip_from 103.21.244.0/22; # Cloudflare example range
+ set_real_ip_from 173.245.48.0/20;
+ # Layer 2: Your ALB/NLB subnet
+ set_real_ip_from 10.100.0.0/16;
+ real_ip_header X-Forwarded-For;
+ real_ip_recursive on;
+ # Log the RESOLVED IP, not the raw header
+ log_format main '$remote_addr - $remote_user [$time_local] "$request" '
+ '$status $body_bytes_sent "$http_referer" "$http_user_agent"';
- # Never log $http_x_forwarded_for directly in security-sensitive logs
}
💡 Tired of pasting proprietary configs into ChatGPT? Generic AI tools log your company's ARNs, DB strings, and private keys. StackEngine is a zero-backend, pure Client-Side WASM utility. Drop your failing config into the sandbox above. We redact your secrets locally in the browser and auto-generate the refactored code using your own API key.
Prevention in CI/CD
1. Gixy — Nginx Static Security Analyzer
# Install and run in your pipeline
pip install gixy
gixy /etc/nginx/nginx.conf
# Catches: http_splitting, add_header_redefinition, ssrf, host_spoofing
# Does NOT catch XFF issues natively — write a custom rule (see below)
2. OPA/Conftest Policy for XFF Enforcement
# policy/nginx_xff.rego
package nginx.security
deny[msg] {
input.http.limit_req_zone.key == "$http_x_forwarded_for"
msg := "CRITICAL: limit_req_zone must not key on $http_x_forwarded_for. Use $binary_remote_addr after real_ip resolution."
}
deny[msg] {
not input.http.real_ip_recursive == "on"
msg := "real_ip_recursive must be 'on' when using X-Forwarded-For with multiple proxy hops."
}
# In CI (GitHub Actions, GitLab CI)
nginx -t -c nginx.conf # syntax check
conftest test nginx.conf --policy policy/
3. Automate CDN IP Range Updates
Cloudflare, AWS CloudFront, and GCP publish their egress IP ranges via API. If your set_real_ip_from list goes stale, legitimate IPs stop being resolved correctly — or worse, new CDN IPs become untrusted and attacker XFF values survive. Run a weekly cron to regenerate the Nginx include file from provider IP range APIs and reload Nginx (nginx -s reload).
4. Integration Test in Staging
# Verify spoofed XFF is NOT trusted
curl -H "X-Forwarded-For: 127.0.0.1" https://staging.example.com/api/admin
# Expected: 403. If 200 → your config is broken.
# Verify real client IP is logged correctly
curl https://staging.example.com/api/health
tail -1 /var/log/nginx/access.log | awk '{print $1}'
# Must be YOUR actual egress IP, not the LB IP.