Sign in Subscribe

Migrating Terraform Backend to Amazon S3 and DynamoDB: A Guide

Utpal Nadiger

• 2 min read

Terraform’s recent move to the resources under management pricing model has frustrated a lot of users. As one user on this reddit thread remarked:

The pricing on Terraform literally just jumped to almost same as we are paying for the AWS resources it manages.We’re on the free plan, with only 2 devs — were looking at moving to the paid plan this month which was going to be $20/user/month — but now we’re looking at a ~ $250 monthly bill for our dev, test and prod environments. We’re only paying $300 for the AWS resources.

Migrating your backend from Terraform Cloud to an infrastructure based on Amazon S3 and DynamoDB may help save you some $$ and help extend that end of runway.

It involves configuring Terraform to use these services for remote state storage. Here’s a guide on how to perform this migration:

  1. Prepare the infrastructure:
  • Create an S3 bucket: Log in to the AWS Management Console and create a new S3 bucket to store your Terraform state files. Choose a unique name for the bucket.
  • Create a DynamoDB table: In the AWS Management Console, create a new DynamoDB table to manage Terraform state locking. Provide a table name and a primary key (e.g., “LockID” of type String).

2. Update Terraform configuration:

  • Open your Terraform configuration files (e.g., main.tf, backend.tf).
  • Remove the existing Terraform Cloud backend configuration.
  • Add a new backend configuration to use S3 and DynamoDB. Here’s an example:

NOTE: Replace <your-s3-bucket-name> with the name of the S3 bucket you created in step 1, <aws-region> with the appropriate AWS region, and <your-dynamodb-table-name> with the name of the DynamoDB table you created.

3. Initialize the backend:

  • Open a terminal or command prompt in the directory containing your Terraform configuration files.
  • Run terraform init to initialize the new backend configuration and download any necessary provider plugins.

4. Migrate the existing state:
(Hashicorp Help Center Docs)

  • If you have an existing Terraform state in Terraform Cloud, you’ll need to download it and migrate it to the new backend.
  • Use the terraform state pull command to download the current state from Terraform Cloud.
  • Run terraform init -reconfigure to reconfigure Terraform with the new backend and upload the state to S3 and DynamoDB.

5. Test the new backend

  • Run any Terraform commands (e.g., terraform plan, terraform apply) to ensure the new backend is working correctly.
  • Verify that the state files are stored in the S3 bucket and that the DynamoDB table is being used for state locking.

6. Clean up:

  • If you’re confident that the migration was successful, you can remove the old Terraform Cloud backend configuration and associated state files.

Disclosure: We’re building Digger — an open source GitOps tool for Terraform that reuses the async jobs infrastructure with compute, orchestration, logs, etc of your existing CI/CD. Do feel free to try it out!

(This was originally published on Medium.)

Sign up for more like this.

Enter your email
Subscribe