RAPID INCIDENT TRIAGE & ACTION PLAN•Security
01. Error Signature
PostgreSQL SCRAM-SHA-256 Auth Failure Debugger
02. Root Cause Vector

PostgreSQL rejects login when the server demands SCRAM-SHA-256 but the client or pg_hba.conf is configured for MD5, causing immediate app...

03. Rapid Prescription

Enforce consistent lock order (e.g. ORDER BY id FOR UPDATE) or use non-blocking SKIP LOCKED.

Zero-Trust Client IDE/ Security

PostgreSQL SCRAM-SHA-256 Auth Failure Debugger

PostgreSQL rejects login when the server demands SCRAM-SHA-256 but the client or pg_hba.conf is configured for MD5, causing immediate application downtime.

SYSTEM STATUS
CREDITS: 0 / 3
source.yaml(57L)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# legacy-db-deployment.yaml
# TODO: Migrate to new K8s cluster after PCI audit
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: legacy-postgres-app
  namespace: pci-secure-zone
  labels:
    app: legacy-db-proxy
    tier: backend
spec:
  replicas: 2
  selector:
    matchLabels:
      app: legacy-db-proxy
  template:
    metadata:
      labels:
        app: legacy-db-proxy
    spec:
      containers:
      - name: app-container
        image: my-registry.internal/legacy-api:v3.7.1-stale
        env:
        - name: DB_HOST
          value: "postgres-master.pci-secure-zone.svc.cluster.local"
        - name: DB_USER
          value: "appuser"
        # VULNERABILITY: SCRAM-SHA-256 password passed, but pg_hba.conf on server only allows md5 for this host.
        - name: DB_PASSWORD
          value: "Ak3jF9!x8@qP#mZ$vL2%cR7&bN1^yT5*wU6"  # High-entropy dummy secret
        - name: AWS_ACCESS_KEY_ID
          value: "AKIAIOSFODNN7EXAMPLE"
        - name: AWS_SECRET_ACCESS_KEY
          value: "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
        - name: STRIPE_API_KEY
          value: "sk_live_4eC39HqLyjWDarjtT1zdp7dc"
        command: ["/bin/sh"]
        args:
        - "-c"
        - |
          # Workaround for TLS handshake timeout (remove after Q3)
          echo '-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAz7X8... (truncated dummy key)\n-----END RSA PRIVATE KEY-----' > /tmp/dummy.pem
          # TODO: Fix auth method mismatch - dev says SCRAM, ops says md5 in pg_hba
          python3 /app/connect.py
      imagePullSecrets:
      - name: legacy-registry-creds
---
# Old config snippet kept for reference (commented out)
# apiVersion: v1
# kind: ConfigMap
# metadata:
#   name: postgres-client-config
# data:
#   pgpass: "postgres-master:5432:*:appuser:OldPass123!"
AI Audit Console
> Waiting for pre-flight audit...
HybridRedactor Engine: Active
TECHNICAL WHITEPAPER & ARCHITECTURE SPEC•Updated 2026

Fixing PostgreSQL SCRAM-SHA-256 Auth Failure: password authentication failed for user with pg_hba.conf md5

Threat/Impact Level: HIGH | Downtime Risk: HIGH | Time to Fix: 5–15 mins


TL;DR

  • What broke: PostgreSQL 14+ defaults password_encryption to scram-sha-256, but your pg_hba.conf still specifies md5 — or vice versa — causing a handshake mismatch that hard-blocks every login for appuser.
  • How to fix it: Align pg_hba.conf auth method with password_encryption, then re-hash the password (ALTER USER appuser WITH PASSWORD '...') so the stored verifier matches the negotiated method.
  • Fast path: Use our Client-Side Sandbox above to auto-refactor this — paste your pg_hba.conf and get a corrected diff without sending secrets to any server.

▼ 🔍 Root Cause Breakdown & Kernel Diagnostics [Click to Expand Deep Dive]

The Incident (What Does the Error Mean?)

Raw error from PostgreSQL logs / client:

FATAL:  password authentication failed for user "appuser"
DETAIL:  Connection matched pg_hba.conf line 87: "host all appuser 10.0.0.0/8 md5"

Or the inverse — client library error:

ERROR:  auth method "scram-sha-256" requires libpq >= 10
FATAL:  password authentication failed for user "appuser"

What is actually failing: PostgreSQL's authentication handshake is a two-part contract. The server advertises an auth method (from pg_hba.conf). The stored password verifier in pg_authid must have been hashed with a compatible algorithm at CREATE USER / ALTER USER time. If these two are misaligned — or if the client library is too old to speak SCRAM — the handshake aborts before a single query runs. Every connection attempt fails. Your app is down.


▼ 🔍 Detailed Impact & Blast Radius Analysis [Click to Expand Deep Dive]

The Attack Vector / Blast Radius

This is not just an inconvenience — it is a security regression in disguise.

Why MD5 is the problem, not the fix: MD5 password hashing for PostgreSQL auth (RFC: md5 in pg_hba.conf) stores a salted MD5 hash of password + username. MD5 is cryptographically broken. A leaked pg_authid table — via a SQL injection COPY TO attack, a misconfigured pg_dump, or a compromised replica — exposes password hashes that can be cracked offline with commodity hardware in minutes using hashcat.

SCRAM-SHA-256 (RFC 5802) is the mandated replacement because:

  • The verifier stored in pg_authid is not directly usable for replay attacks.
  • The protocol performs mutual authentication — the client proves it knows the password without transmitting it in a recoverable form.
  • Channel binding (scram-sha-256-plus) ties the auth to the TLS session, defeating MITM.

Blast radius of leaving MD5 in place:

  1. Leaked replica or backup → offline crack of all MD5-hashed passwords.
  2. If appuser password is reused elsewhere (common in legacy apps) → lateral movement.
  3. PCI-DSS, SOC 2, and HIPAA auditors will flag MD5 auth as a finding.

Blast radius of the mismatch itself:

  • 100% connection failure for the affected user/host combination.
  • If this is a connection pool (PgBouncer, RDS Proxy), the pool exhausts retries and cascades into app-layer 500s within seconds.

How to Fix It

Step 1 — Diagnose the stored hash type

-- Run as superuser
SELECT rolname, rolpassword FROM pg_authid WHERE rolname = 'appuser';
  • Starts with md5 → password was hashed with MD5. Must be re-set.
  • Starts with SCRAM-SHA-256$ → hash is correct; the problem is pg_hba.conf.
  • NULL → no password set at all.

Step 2 — Check server default

SHOW password_encryption;
-- Should return: scram-sha-256

Basic Fix — Align pg_hba.conf and re-hash

pg_hba.conf correction:

# /etc/postgresql/15/main/pg_hba.conf

- host    all    appuser    10.0.0.0/8    md5
+ host    all    appuser    10.0.0.0/8    scram-sha-256

Force re-hash of the stored password (critical — do not skip):

- -- Password verifier currently stored as MD5 in pg_authid
- -- ALTER USER appuser WITH PASSWORD 'yourpassword';  ← run when password_encryption=md5

+ -- Ensure server will hash with SCRAM before running ALTER USER
+ SET password_encryption = 'scram-sha-256';
+ ALTER USER appuser WITH PASSWORD 'yourpassword';

Then reload (no restart needed for pg_hba.conf):

sudo systemctl reload postgresql
# or in psql:
SELECT pg_reload_conf();

Enterprise Best Practice — Enforce at the cluster level + secret rotation

postgresql.conf — lock the default:

- #password_encryption = md5
+ password_encryption = scram-sha-256

Use channel binding if TLS is enforced (highest security):

- host    all    appuser    10.0.0.0/8    scram-sha-256
+ hostssl all    appuser    10.0.0.0/8    scram-sha-256

Rotate via Vault / AWS Secrets Manager (never hardcode):

- # App reads password from environment variable set in Dockerfile
- ENV DB_PASSWORD=plaintextpassword

+ # App reads password from Secrets Manager at runtime
+ # aws secretsmanager get-secret-value --secret-id prod/appuser/db
+ # Inject via ECS task role — no static secret in image

Verify the fix:

psql "host=yourhost dbname=yourdb user=appuser sslmode=require" -c "SELECT current_user;"
# Expected: appuser

💡 Tired of pasting proprietary infrastructure configs into public LLMs? Generic AI interfaces log your company's ARNs, connection strings, and private keys. StackEngine enforces client-side zero-trust boundaries—the sandbox above executes 100% locally in your browser. Drop your broken manifest above to redact secrets locally and synthesize deterministic patch drafts with your own API key.

(For CI/CD pipeline automation, install via npm i -g stackengine).


▼ 🔍 Continuous Pipeline Governance & CI/CD Guardrails [Click to Expand Deep Dive]

Prevention in CI/CD

1. Checkov — scan Terraform RDS / PostgreSQL modules:

# .checkov.yml
checks:
  - CKV_AWS_129   # RDS: ensure IAM auth enabled (supplements password auth)
  - CKV2_AWS_60  # RDS: enforce SSL
# Add custom check: flag any pg_hba.conf committed to repo with 'md5'

2. OPA/Conftest policy — block md5 in pg_hba.conf committed to IaC:

package postgresql

deny[msg] {
  line := input.pg_hba_lines[_]
  contains(line, "md5")
  msg := sprintf("pg_hba.conf line uses deprecated md5 auth: %v", [line])
}

3. GitHub Actions — validate pg_hba.conf on PR:

- name: Reject md5 auth in pg_hba.conf
  run: |
    if grep -rE '\bmd5\b' ./postgres/pg_hba.conf; then
      echo "ERROR: md5 auth method detected. Use scram-sha-256."
      exit 1
    fi

4. Terraform aws_db_parameter_group — enforce password_encryption:

 resource "aws_db_parameter_group" "postgres" {
   family = "postgres15"
+  parameter {
+    name  = "password_encryption"
+    value = "scram-sha-256"
+    apply_method = "immediate"
+  }
 }

5. Monitor for auth failures in production:

# CloudWatch Logs Insights (RDS) or grep for on-prem
filter @message like /password authentication failed/
| stats count(*) by bin(5m)
# Alert if > 5 failures/min — may indicate credential stuffing or misconfigured deploy

Official Grounding & Normative Specifications

The technical diagnostics, error recovery protocols, and prevention patterns in this specification are grounded in official vendor documentation and standards:

Diagnostic & Architectural FAQ

Q1.Why does PostgreSQL 14+ suddenly break MD5 auth that worked on PostgreSQL 13?
PostgreSQL 14 changed the default value of `password_encryption` from `md5` to `scram-sha-256`. Any user whose password was set before the upgrade still has an MD5 verifier stored in `pg_authid`. If you also updated `pg_hba.conf` to `scram-sha-256`, the handshake fails because the stored hash type doesn't match the negotiated method. This is a critical misalignment between your declared security posture and your actual runtime state—a gap that manual intervention is prone to miss. Drop the broken manifest into the StackEngine Local Sandbox above to automatically compile a zero-trust fix that reconciles this without exposing your database to manual, error-prone SQL commands.
Q2.Can I just change pg_hba.conf back to md5 to restore service immediately?
Yes, reverting `pg_hba.conf` to `md5` will restore connectivity as a break-glass measure—but you are reintroducing a cryptographically broken auth method, creating a severe security vulnerability. This manual 'fix' expands your blast radius, leaving your entire database exposed to trivial attacks. Schedule a maintenance window to re-hash all passwords with SCRAM and flip back to `scram-sha-256` within 24 hours. Do not treat MD5 as a permanent solution in any environment handling sensitive data. Drop the broken manifest into the StackEngine Local Sandbox above to automatically compile a zero-trust fix that enforces SCRAM without the risk of human error or configuration drift.
Q3.My client library throws 'auth method scram-sha-256 not supported' — is that a server problem?
No—that is a client-side version problem. SCRAM-SHA-256 requires libpq 10+, JDBC 42.2.0+, psycopg2 2.8+, or node-postgres 8.0+. The server config is correct; you need to upgrade your database driver. Doing this manually across large mono-repos or at scale is a compliance nightmare, risking dependency conflicts and audit failures. Deploy the StackEngine CI/CD Automator (Team Edition) for on-prem, automated pull request generation with Maker-Checker review that enforces driver versioning and security compliance across your entire deployment pipeline under Maker-Checker review.