Federated User EC2 RunInstances Permission Fix

The Incident (What Does the Error Mean?)

Raw error output:

An error occurred (UnauthorizedOperation) when calling the RunInstances operation:
You are not authorized to perform this operation.
Encoded authorization failure message: ...
federated user 'arn:aws:sts::123456789012:assumed-role/MyFederatedRole/session-name'
is not authorized to perform: ec2:RunInstances on resource: arn:aws:ec2:us-east-1:...

Immediate consequence: Every pipeline, autoscaling event, or manual launch hitting this role returns HTTP 403. If this role backs an autoscaling group or EKS node pool, your fleet cannot scale out. Deployments stall. On-call gets paged. The root cause is always one of three things:

1. The trust policy on the IAM Role allows the federated IdP to assume it, but the permission policy is missing ec2:RunInstances.
2. A Service Control Policy (SCP) at the AWS Organizations level has an explicit Deny on EC2 actions.
3. A permission boundary is attached to the role that doesn't include ec2:RunInstances, silently capping what the role can do regardless of attached policies.

The Attack Vector / Blast Radius

This is a least-privilege enforcement gap — but it cuts both ways.

Why it's a production risk: Federated roles are typically broad — they're assumed by humans via SSO or by CI/CD systems via OIDC. If someone "fixes" this by attaching ec2:* or AmazonEC2FullAccess, they've handed a potentially compromised session token the ability to launch GPU instances in any region, exfiltrate data via user-data scripts, or pivot into a VPC. A stolen STS token with ec2:RunInstances + iam:PassRole is a full account compromise vector — an attacker can launch an instance, attach a privileged instance profile, and extract credentials from the metadata endpoint.

SCP blast radius: If the Deny is at the SCP level, every role in the account/OU is affected. Fixing only this role's policy will not resolve the error — you'll burn 30 minutes wondering why your policy looks correct.

Checklist before touching policies:

# Simulate the exact call to identify which policy layer is blocking
aws iam simulate-principal-policy \
  --policy-source-arn "arn:aws:sts::123456789012:assumed-role/MyFederatedRole/session" \
  --action-names "ec2:RunInstances" \
  --resource-arns "arn:aws:ec2:us-east-1::*"

# Decode the authorization failure message from the error
aws sts decode-authorization-message --encoded-message <encoded_string>

Prevention in CI/CD

1. Checkov — scan IAM policies in Terraform before apply:

checkov -d ./iam --check CKV_AWS_40,CKV_AWS_274
# CKV_AWS_274 flags ec2:RunInstances without condition constraints

2. OPA/Conftest policy — block wildcard EC2 launch in policy-as-code:

# policy/ec2_run_instances.rego
package aws.iam

deny[msg] {
  stmt := input.Statement[_]
  stmt.Effect == "Allow"
  action := stmt.Action[_]
  action == "ec2:RunInstances"
  stmt.Resource == "*"
  not stmt.Condition
  msg := "ec2:RunInstances on Resource:* requires Condition constraints (region, instanceType, VPC)"
}

3. Terraform — always attach a permission boundary to federated roles:

resource "aws_iam_role" "federated" {
  name                 = "MyFederatedRole"
  assume_role_policy   = data.aws_iam_policy_document.trust.json
  # NEVER omit this in prod
  permissions_boundary = aws_iam_policy.boundary.arn
}

4. AWS Config Rule — continuous compliance:

aws configservice put-config-rule --config-rule '{
  "ConfigRuleName": "iam-no-wildcard-ec2-run",
  "Source": {
    "Owner": "CUSTOM_LAMBDA",
    "SourceIdentifier": "arn:aws:lambda:...:function:CheckEC2RunInstancesScope"
  }
}'

Set up a CloudTrail → EventBridge → Lambda pipeline to alert on any CreatePolicy or AttachRolePolicy event that introduces ec2:RunInstances without condition keys. Catch it before it merges, not during a 2am outage.