Terraform Remote State with S3

Terraform Remote State with S3

Terraform Remote State with S3 and DynamoDB — NAXS Labs

Terraform Remote State
with S3 and DynamoDB

The previous post set up OPA policy enforcement in a GitLab CI/CD pipeline. One problem remained: local state. Here’s how we fixed it.

All posts

In the previous post we built a GitLab CI/CD pipeline that runs OPA policy checks against a Terraform plan before any deployment can happen. The pipeline worked — but there was a deferred problem. Terraform state was stored locally, which meant tofu destroy from the pipeline had no state file to read and couldn’t clean up resources. That’s a real operational gap, and it maps to a real compliance gap: state files contain resource configurations, ARNs, and sometimes sensitive values. Leaving them on a local machine or in a repo is not a defensible posture.

This post covers adding a remote S3 backend with DynamoDB locking to close both problems.

Why Local State Is a Problem

Terraform state is the source of truth for what infrastructure exists. Without it, Terraform doesn’t know what it deployed and can’t manage it. Local state means that source of truth lives on whoever’s machine ran the last apply — which breaks pipeline-driven workflows entirely and creates an availability problem if that machine disappears.

Beyond the operational issue, local state is a security finding. State files can contain plaintext sensitive values depending on what resources are managed. Storing them outside of version-controlled, encrypted, access-controlled storage means you have sensitive infrastructure data with no audit trail and no access controls. SC-28 (protection of information at rest) applies here the same way it applies to any other sensitive data store.

The Bootstrap Problem

You can’t use Terraform to create the S3 bucket that will store Terraform state — because that Terraform run has no remote backend yet. It’s a chicken-and-egg problem. The standard solution is a one-time manual bootstrap: create the state bucket and lock table via CLI, then configure Terraform to use them going forward.

# Create the state bucket
aws s3api create-bucket \
  --bucket naxslabs-tf-state \
  --region us-east-1

# Enable versioning — state history and rollback
aws s3api put-bucket-versioning \
  --bucket naxslabs-tf-state \
  --versioning-configuration Status=Enabled

# Enable encryption at rest
aws s3api put-bucket-encryption \
  --bucket naxslabs-tf-state \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "AES256"
      }
    }]
  }'

# Block all public access
aws s3api put-public-access-block \
  --bucket naxslabs-tf-state \
  --public-access-block-configuration \
    "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

# Create DynamoDB lock table
aws dynamodb create-table \
  --table-name naxslabs-tf-state-lock \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --region us-east-1

Versioning is worth enabling because it gives you state history so if a bad apply corrupts state you can roll back to a previous version. Encryption and public access blocking are not optional; state files belong behind both.

Cost

A few KB of state data in S3 and a handful of DynamoDB lock operations per month costs effectively nothing — well under $0.05 at any reasonable usage volume. The DynamoDB table uses on-demand billing so there’s no hourly charge for it sitting idle.

Backend Configuration

With the bucket and lock table created, add the backend block to main.tf:

terraform {
  required_providers {
    aws = {
      source  = "opentofu/aws"
      version = "~> 6.49.0"
    }
    random = {
      source  = "opentofu/random"
      version = "~> 3.9.0"
    }
  }

  backend "s3" {
    bucket         = "naxslabs-tf-state"
    key            = "opa-terraform-aws/s3/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "naxslabs-tf-state-lock"
  }
}

The key value is the path inside the bucket where this module’s state file lives. The convention repo/module/terraform.tfstate means each module in each repo gets its own key — no collisions when the EC2 module and the Azure repo get added later.

After adding the backend block, run tofu init. Terraform detects the new backend configuration and prompts to migrate existing local state to S3. Type yes. From that point on, all state reads and writes go through the bucket.

How Locking Works

The DynamoDB table prevents concurrent operations from corrupting state. When any Terraform operation that modifies state begins — plan, apply, destroy — it writes a lock record to the DynamoDB table with a unique ID. If a second operation tries to start while the lock exists, it fails immediately with a lock error rather than proceeding and potentially writing conflicting state. When the operation completes, the lock record is deleted.

In a pipeline context this matters when multiple pipeline runs could overlap — a manual deploy triggered while a previous run is still in progress. Without locking, both runs read the same state, make independent changes, and the last write wins. With locking, the second run fails fast and the pipeline output tells you why.

The Pipeline End-to-End

With remote state in place the full pipeline now works without manual intervention at any stage:

  • Validate — runs terraform init (pulls remote state) and terraform validate
  • Plan — generates a plan against current remote state, outputs tfplan.json as a pipeline artifact
  • Policy — Conftest evaluates tfplan.json against OPA policies; blocks deployment if any deny rules fire
  • Deploy — manual trigger; applies the plan, writes updated state to S3
  • Destroy — manual trigger; reads state from S3, destroys all managed resources, empties the state file

Before remote state, destroy from the pipeline failed silently — no local state file meant no resources to destroy. Now the pipeline is the authoritative path for both provisioning and teardown, which is what infrastructure-as-code is supposed to look like.

Control Mapping

NIST SC-28 NIST AU-2 NIST CM-6 NIST CP-9

SC-28 (Protection of Information at Rest) — AES-256 encryption on the state bucket satisfies at-rest protection for the infrastructure configuration data the state file contains. Public access blocking ensures the bucket cannot be inadvertently exposed regardless of IAM policy gaps.

AU-2 (Event Logging) — S3 versioning creates an implicit audit trail of every state change. Each apply that modifies state produces a new object version, so the full history of infrastructure state is preserved and recoverable.

CM-6 (Configuration Settings) — the backend configuration itself is a configuration control: encryption enforced at the backend level means it applies regardless of who runs Terraform or from where. It’s not dependent on the operator remembering to enable it.

CP-9 (System Backup) — versioning on the state bucket is a form of backup for the infrastructure state. If a bad apply produces corrupt or incorrect state, any previous version can be restored directly from S3 without needing to reconstruct state manually.


The full repo is at git.naxslabs.com/darnell/opa-terraform-aws.

NAXS Labs
Logo