Skip to main content

Migrating Kubernetes Terraform Resources from Non-Versioned to Versioned Resources in Coder

Updated

Overview

This article provides step-by-step instructions for migrating existing Coder workspaces that use Kubernetes Terraform resources from their non-versioned resource types (e.g., kubernetes_deployment) to their versioned counterparts (e.g., kubernetes_deployment_v1).

This migration capability was added in Coder versions 2.27.10, 2.28.7, and 2.29.2 through the introduction of the --no-build flag for state management operations, which allows you to update workspace state without triggering a workspace build.

Related Pull Request: https://github.com/coder/coder/pull/21374

Prerequisites:

Before beginning this migration process, ensure you have:

  • Coder deployment upgraded to version 2.27.10, 2.28.7, or 2.29.2 (or later)
  • Coder CLI upgraded to match the server version
  • Access to pull workspace state and templates
  • Terraform CLI installed on your local system
  • Access to modify and push templates

Migration Steps

Step 1: Upgrade Your Coder Deployment

Upgrade your Coder deployment to one of the following versions that include the state push functionality:

  • 2.27.10
  • 2.28.7
  • 2.29.2 (or later)

Step 2: Pull the Workspace State

Pull the current state of the workspace you want to migrate and save it locally:

 
bash
coder state pull <workspace-name> > <workspace-name>.tfstate

This command saves the workspace state to a file named <workspace-name>.tfstate on your local system.

Step 3: Verify Current Resources

List the resources in the state file to confirm they are using non-versioned resource types:

 
bash
terraform state list -state=<workspace-name>.tfstate

Review the output to identify all non-versioned Kubernetes resources that need migration.

Step 4: Pull the Template

Download the corresponding template from your Coder deployment:

 
bash
coder templates pull <template_name> --org <org_name>

Step 5: Update Template Configuration

Navigate to the template directory and modify the Terraform configuration files:

  1. Change resource names to include the version suffix:
    • Before: kubernetes_deployment
    • After: kubernetes_deployment_v1
  2. Apply this change to all non-versioned Kubernetes resources (deployments, PVCs, secrets, services, etc.)
  3. Update all references to these resources throughout your Terraform configuration to use the new versioned names

Step 6: Remove Old Resources from State

Remove each non-versioned resource from the state file:

 
bash
terraform state rm -state=<workspace-name>.tfstate kubernetes_deployment.main

Repeat this command for all non-versioned Kubernetes resources (PVCs, secrets, services, etc.).

Step 7: Verify Resource Removal

Confirm that all non-versioned Kubernetes resources have been removed from the state:

 
bash
terraform state list -state=<workspace-name>.tfstate

Ensure no non-versioned resources remain in the output.

Step 8: Import Resources with Versioned Names

While in the template directory, import each resource using its new versioned name:

 
bash
terraform import -state=<workspace-name>.tfstate \
  -var="use_kubeconfig=true" \
  -var="namespace=coder" \
  'kubernetes_persistent_volume_claim_v1.home' \
  coder/coder-13e44247-8bce-4000-99c2-edd243fd9856-home

Important notes:

  • Adjust variables, resource addresses, and resource IDs to match your specific deployment
  • Repeat this import command for all Kubernetes resources being migrated
  • For resources using meta-arguments like count: Include the index in the resource address
    • ✅ Correct: kubernetes_deployment_v1.main[0]
    • ❌ Incorrect: kubernetes_deployment_v1.main
    • The [0] index indicates you're importing the first instance of the resource created by count

Step 9: Verify Version Compatibility

Before proceeding, confirm that both your CLI and server are running compatible Coder versions (2.27.10, 2.28.7, 2.29.2, or later). The next step requires the --no-build flag, which is only available in these versions.

Step 10: Push Updated State

Push the modified state back to Coder without triggering a workspace build:

 
bash
coder state push --no-build <workspace-name> <workspace-name>.tfstate

The --no-build flag ensures that the state is updated without triggering a workspace build or recreation of resources.

Step 11: Update the Template

Push the updated template configuration to Coder:

 
bash
coder templates push <template-name>

Note: You can perform this step earlier in the process, but do not allow users to update and restart their workspaces until the state migration is complete.

Step 12: Update Workspaces

After the template is updated, users will be prompted to update and restart their workspaces. Because the state has been properly migrated, this update will not delete existing resources such as PVCs.

Important Considerations

  • Backup: Consider backing up your workspace state files before beginning the migration
  • Testing: Test this migration process on a non-production workspace first
  • Downtime: Plan for potential workspace downtime during the migration
  • Resources: Ensure all Kubernetes resources (deployments, PVCs, secrets, services, etc.) are migrated to their versioned counterparts
  • Count/For-Each: Pay special attention to resources using count or for_each meta-arguments when importing

Troubleshooting

If you encounter issues during migration:

  1. Verify both CLI and server versions match and support the --no-build flag
  2. Ensure all non-versioned resources were removed from state before importing
  3. Confirm resource IDs and names match exactly when importing
  4. Check that all references to resources in the template have been updated to use versioned names

Additional Resources

Related articles