Terraform Route53 Record Conflict Resolver

The Incident (What Does the Error Mean?)

Raw error from terraform apply:

Error: [ERR]: Error building changeset: InvalidChangeBatch:
[Tried to create resource record set [name='api.example.com.',
type='A'] but it already exists]
        status code: 400, request id: 4f1a2b3c-...

Route53 operates on atomic change batches. When Terraform submits a CREATE action for a record that already exists — regardless of whether the values are identical — Route53 rejects the entire batch with a 400. This means every other record change in that same apply run also failed. Your DNS changes are not partially applied; they are fully blocked.

Immediate consequence: New infrastructure (ALB, CloudFront, EC2) is live but unreachable by name. Deployment is in a broken half-state.

The Attack Vector / Blast Radius

This is a state drift problem, not a code bug. The record exists in AWS via one of these vectors:

1. Manual console creation — someone hotfixed DNS during a previous incident and never cleaned up.
2. Orphaned state — the record was once managed by Terraform in a different workspace, a deleted state file, or a different repo.
3. Duplicate resource blocks — two aws_route53_record resources in your codebase resolve to the same zone_id + name + type composite key.
4. Another IaC tool — CDK, CloudFormation, or a Pulumi stack owns this record and Terraform has no knowledge of it.

Blast radius: Until resolved, any terraform apply touching this hosted zone is fully blocked. In a monorepo or root module that manages dozens of records, a single conflict freezes all DNS automation. Blue/green cutover, certificate validation records, and health-check failover records all queue behind this failure.

Prevention in CI/CD

1. Drift Detection on a Schedule

Run terraform plan in read-only mode on a cron in CI. A non-zero exit code on plan means drift has occurred before your next apply hits it.

# .github/workflows/drift-detection.yml
- name: Detect Route53 Drift
  run: |
    terraform init
    terraform plan -detailed-exitcode -target=aws_route53_record.api_record
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}

2. Checkov Policy — Block `allow_overwrite` in Permanent Configs

# checkov custom check: no permanent allow_overwrite
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck

class Route53NoAllowOverwrite(BaseResourceCheck):
    def __init__(self):
        super().__init__(
            name="Ensure allow_overwrite is not set on Route53 records",
            id="CKV_CUSTOM_R53_001",
            supported_resources=["aws_route53_record"]
        )
    def scan_resource_conf(self, conf):
        return conf.get("allow_overwrite", [False])[0] is not True

3. Enforce State Hygiene with Terraform Sentinel (TFC/TFE)

# sentinel policy: require all route53 records to exist in state before apply
import "tfstate"

main = rule {
  all tfstate.resources as _, r {
    r.type is not "aws_route53_record" or
    r.tainted is false
  }
}

4. Tag Convention to Track IaC Ownership

Route53 records do not support resource tags, but hosted zones do. Tag every zone with managed-by = terraform and the workspace name. When a record appears without a corresponding Terraform resource, your drift alarm fires before it causes a conflict.

 resource "aws_route53_zone" "primary" {
   name = "example.com"
+  tags = {
+    managed-by        = "terraform"
+    terraform-workspace = terraform.workspace
+    team              = "platform"
+  }
 }