Glue Lake Formation CreateDatabase Permission Fix

The Incident (What Does the Error Mean?)

You hit this at 2 AM during a pipeline deploy or a Terraform apply:

An error occurred (AccessDenied) when calling the CreateDatabase operation:
  User: arn:aws:sts::123456789012:assumed-role/MyGlueRole/session
  is not authorized to perform: glue:CreateDatabase
  on resource: arn:aws:glue:us-east-1:123456789012:catalog
  with an explicit deny

The phrase "with an explicit deny" is the tell. This is not a missing IAM policy. This is Lake Formation's resource-level authorization layer issuing an active denial. Once LF is enabled on your account (either manually or via AWS Organizations SCPs), it intercepts all Glue Data Catalog API calls. Your IAM policy is irrelevant at this point — LF has its own permission model layered on top, and the principal has zero LF grants.

Immediate consequence: Any ETL job, Glue crawler, Terraform Glue resource, or data pipeline that calls CreateDatabase is dead in the water. CI/CD pipelines fail silently or with cryptic errors if your tooling doesn't surface the full AWS error response.

The Attack Vector / Blast Radius

This misconfiguration cuts both ways — it's a security control working as intended that becomes an operational incident when not provisioned correctly.

Why this is dangerous if you work around it wrong:

The instinctive "fix" engineers reach for is re-enabling Use only IAM access control in Lake Formation settings, or granting lakeformation:* to the role. Both are catastrophic:

• Re-enabling IAM-only mode retroactively removes all fine-grained LF column/row/table permissions across your entire data lake — every downstream Athena query, Redshift Spectrum job, and EMR cluster that relied on LF grants breaks simultaneously.
• Granting lakeformation:* to a service role means that role can now grant itself access to any table in your data catalog, exfiltrate schemas, and register arbitrary S3 paths as data lake locations — a full lateral movement path inside your data perimeter.

Blast radius of the root cause:

• All Glue jobs, crawlers, and Terraform modules using this role fail.
• If this role is shared across environments (common anti-pattern), prod, staging, and dev pipelines all halt simultaneously.
• If the role is attached to a Lambda or Step Functions state machine, downstream orchestration silently skips database provisioning steps, causing data quality failures hours later.

Prevention in CI/CD

1. Checkov — Catch Missing LF Grants in Terraform Plans

Add this to your .checkov.yml or pipeline step:

# .checkov.yml
checks:
  - CKV_AWS_97  # Ensure Glue Data Catalog encryption is enabled
  # Custom check: ensure every aws_glue_catalog_database has a paired
  # aws_lakeformation_permissions resource in the same module

For a custom Checkov policy:

# checkov/custom_checks/check_lf_grant_paired_with_glue_db.py
from checkov.terraform.checks.resource.base_resource_check import BaseResourceCheck
from checkov.common.models.enums import CheckResult, CheckCategories

class GlueDBHasLFGrant(BaseResourceCheck):
    def __init__(self):
        name = "Ensure aws_glue_catalog_database has paired LF permissions"
        id = "CKV_CUSTOM_LF_001"
        supported_resources = ["aws_glue_catalog_database"]
        categories = [CheckCategories.IAM]
        super().__init__(name=name, id=id, categories=categories,
                         supported_resources=supported_resources)

    def scan_resource_conf(self, conf):
        # Flag for manual review — LF grants must be verified in plan
        return CheckResult.FAILED

2. OPA / Conftest — Enforce LF Grant Existence in Terraform Plans

# policies/lakeformation_grant_required.rego
package terraform.lakeformation

deny[msg] {
  resource := input.resource_changes[_]
  resource.type == "aws_glue_catalog_database"
  resource.change.actions[_] == "create"
  
  # Check no paired lakeformation_permissions resource targets this database
  not lf_grant_exists(resource.address)
  
  msg := sprintf(
    "[LF-001] aws_glue_catalog_database '%v' has no paired aws_lakeformation_permissions grant. CreateDatabase will fail if Lake Formation is enabled.",
    [resource.address]
  )
}

lf_grant_exists(db_address) {
  resource := input.resource_changes[_]
  resource.type == "aws_lakeformation_permissions"
  contains(resource.change.after.database[_].name, db_address)
}

3. Pipeline Gate — Validate LF Settings Before Apply

#!/bin/bash
# scripts/validate_lakeformation.sh — run before terraform apply in CI

set -euo pipefail

LF_SETTINGS=$(aws lakeformation get-data-lake-settings --region "${AWS_REGION}")
IAM_ALLOWED=$(echo "$LF_SETTINGS" | jq -r '.DataLakeSettings.CreateDatabaseDefaultPermissions | length')

if [ "$IAM_ALLOWED" -gt 0 ]; then
  echo "[WARN] Lake Formation has default catalog permissions set. Verify your Glue role has explicit LF grants before deploying."
fi

# Verify the deploying role has CREATE_DATABASE grant
ROLE_ARN=$(aws sts get-caller-identity --query Arn --output text)
PERMS=$(aws lakeformation list-permissions \
  --principal DataLakePrincipalIdentifier="$ROLE_ARN" \
  --resource '{"Catalog": {}}' \
  --query 'PrincipalResourcePermissions[].Permissions[]' \
  --output text)

if [[ ! "$PERMS" =~ "CREATE_DATABASE" ]]; then
  echo "[FATAL] Role $ROLE_ARN lacks CREATE_DATABASE on LF catalog. Apply will fail."
  exit 1
fi

echo "[OK] Lake Formation CREATE_DATABASE grant verified for $ROLE_ARN"

Wire this into your GitHub Actions or GitLab CI as a pre-terraform apply step. It catches the missing grant before your pipeline burns 10 minutes on a Terraform run that will hard-fail at the Glue resource.