Fixing 'Invalid Character in Variable Value' When Using TF_VAR_ in Terraform

Threat/Impact Level: HIGH | Exploitability/Downtime Risk: HIGH | Time to Fix: 5–15 mins

---

TL;DR

• What broke: A TF_VAR_ environment variable contains a shell metacharacter ($, !, #, @, backticks, unescaped quotes, or newlines) that the shell expanded or truncated before Terraform ever saw the value.
• How to fix it: Wrap the export value in single quotes for literal strings, or use a terraform.tfvars file / -var-file to bypass shell interpolation entirely.
• Shortcut: Use our Client-Side Sandbox above to paste your failing export TF_VAR_* block and auto-generate the correctly escaped or file-based equivalent.

---

The Incident (What Does the Error Mean?)

You ran terraform plan or terraform apply and hit one of these:

Error: Invalid character in variable value

  on <value> line 1:
  The value for variable "db_password" contains invalid characters.

or silently worse — no error, but your variable value was truncated at the first ! or $ by bash history expansion or subshell evaluation. Terraform received hunter instead of hunter2!@#Prod.

Immediate consequence: Deployment proceeds with a wrong, truncated, or empty variable. In a database password scenario, the RDS instance is provisioned with a broken credential and your app is dead on arrival.

---

The Attack Vector / Blast Radius

This isn't just a syntax nuisance. The blast radius is significant:

1. Silent truncation is the worst case. Bash history expansion (!) doesn't throw an error — it silently substitutes or drops characters. TF_VAR_db_password=hunter2!prod becomes TF_VAR_db_password=hunter2 with no warning. Your terraform apply succeeds. Your app fails at runtime.

2. CI/CD secret injection amplifies the risk. In GitHub Actions, GitLab CI, or Jenkins, secrets injected as env vars are already masked in logs. When they're also being silently mangled by shell evaluation, you have no visibility into what value Terraform actually received. Debugging this in a 2 AM incident is brutal.

3. HCL string parsing has its own constraints. Even after the shell passes the value correctly, Terraform's HCL parser rejects raw newlines, certain Unicode control characters, and unescaped backslashes in string variable values passed via environment. The error surfaces late — after authentication to the provider has already occurred.

4. Rotation failures. If your secret rotation pipeline exports new credentials via TF_VAR_, a single special character in the rotated secret breaks the entire rotation apply — leaving infrastructure in a half-migrated credential state.

---

How to Fix It (The Solution)

Basic Fix — Single-Quote Your Exports

The immediate fix for interactive shells and simple CI steps: use single quotes, which prevent all shell interpolation.

# Shell export — the bad way
- export TF_VAR_db_password=hunter2!@#Prod$2024

# Shell export — the correct way (single quotes = literal, no expansion)
+ export TF_VAR_db_password='hunter2!@#Prod$2024'

⚠️ Single quotes work for literal values. If your value itself contains a single quote, you must escape it: 'hunter'\''s-password'.

---

Enterprise Best Practice — Eliminate Shell Variable Injection Entirely

For production pipelines, stop using TF_VAR_ for sensitive values altogether. Use a terraform.tfvars file generated at runtime from a secrets manager, or use -var-file. This removes the shell escaping problem at the source.

Option A: Generate a tfvars file from Vault/AWS Secrets Manager at pipeline runtime

# CI step — bad: injecting raw secret into shell env
- export TF_VAR_db_password=$(aws secretsmanager get-secret-value \
-   --secret-id prod/db/password --query SecretString --output text)
- terraform apply

# CI step — good: write to tfvars file, never touch shell env for secrets
+ aws secretsmanager get-secret-value \
+   --secret-id prod/db/password \
+   --query SecretString \
+   --output text > /tmp/secret.txt
+
+ cat > /tmp/sensitive.tfvars <<EOF
+ db_password = "$(cat /tmp/secret.txt)"
+ EOF
+
+ terraform apply -var-file="/tmp/sensitive.tfvars"
+ rm -f /tmp/sensitive.tfvars /tmp/secret.txt

Option B: Use Terraform's native Vault provider or AWS SSM data sources — pull secrets inside HCL, never in shell.

# variables.tf — bad: expecting secret via TF_VAR_
- variable "db_password" {
-   type = string
- }

# main.tf — good: pull secret natively, no shell env involved
+ data "aws_secretsmanager_secret_value" "db" {
+   secret_id = "prod/db/password"
+ }
+
+ locals {
+   db_password = data.aws_secretsmanager_secret_value.db.secret_string
+ }

Option C: Validate TF_VAR_ values before Terraform runs (pre-flight check)

# CI pipeline pre-flight — add before terraform plan
+ python3 -c "
import os, sys
for k, v in os.environ.items():
    if k.startswith('TF_VAR_'):
        if any(c in v for c in ['\n', '\r', '\x00']):
            print(f'FATAL: {k} contains invalid control character')
            sys.exit(1)
print('TF_VAR_ pre-flight: PASS')
"

---

💡 Tired of pasting proprietary configs into ChatGPT? Generic AI tools log your company's ARNs, DB strings, and private keys. StackEngine is a zero-backend, pure Client-Side WASM utility. Drop your failing config into the sandbox above. We redact your secrets locally in the browser and auto-generate the refactored code using your own API key.

---

Prevention in CI/CD

1. tflint — Catches Variable Misconfigurations Pre-Plan

tflint --enable-rule=terraform_required_variables

Won't catch shell escaping issues, but will catch undeclared or misconfigured variable definitions that compound this problem.

2. checkov — Scan for Secrets Leaked into tfvars

checkov -d . --check CKV_SECRET_6

Prevents the inverse problem: hardcoded secrets in .tfvars files committed to git.

3. Pre-commit Hook — Validate TF_VAR_ Exports in Shell Scripts

Add to .pre-commit-config.yaml:

- repo: https://github.com/shellcheck-py/shellcheck-py
  rev: v0.9.0.6
  hooks:
    - id: shellcheck
      args: ["--severity=warning"]

ShellCheck will flag unquoted variable assignments and dangerous interpolation patterns in any .sh scripts that set TF_VAR_ values.

4. OPA/Conftest Policy — Enforce No TF_VAR_ for Sensitive Variable Names

# policy/no_tfvar_secrets.rego
package terraform.tfvar_safety

deny[msg] {
  key := input.env[k]
  startswith(k, "TF_VAR_")
  regex.match(`(?i)(password|secret|token|key|credential)`, k)
  msg := sprintf("TF_VAR_ env injection forbidden for sensitive variable: %s. Use -var-file with secrets manager.", [k])
}

5. GitHub Actions — Enforce Masked Secrets via env: Block, Never Inline

# .github/workflows/terraform.yml
- run: export TF_VAR_db_password=${{ secrets.DB_PASSWORD }} && terraform apply

+ env:
+   TF_VAR_db_password: ${{ secrets.DB_PASSWORD }}
+ run: terraform apply

GitHub Actions injects env: block values without shell evaluation, bypassing the entire metacharacter problem.