MigrationController

Orchestrating application migrations between Kubernetes clusters.

Description

MigrationController facilitates the migration of deployments and services from a source Kubernetes cluster to a target cluster with minimal to zero downtime. It supports various migration strategies and service connection methods to ensure smooth transitions between environments.

Features

• Migration Strategies:

  • Recreate: Terminate all pods in source cluster and create them in target cluster
  • Rolling: Gradually migrate pods one by one from source to target cluster

• Service Connection Methods:

  • Cross Connect: Bidirectional EndpointSlices connecting services in both clusters
  • Proxy: Single proxy service with endpoints pointing to both clusters*

• Health Probes:

  • Liveness and readiness probe support to ensure application health during migration

• Migration Control:

  • Pause/Resume: Ability to pause and resume migrations at any point
  • Rollback: Support for rolling back migrations to the original state

Working

MigrationController works by:

1. Creating equivalent resources in the target cluster
2. Setting up service connectivity between clusters
3. Gradually shifting traffic from source to target according to the chosen strategy
4. Ensuring health checks pass during migration

Edge Cases and Current Limitations

• Deployment migration: The controller currently supports only deployments. Other resources like StatefulSets, DaemonSets, etc., are not yet supported.
• Looping traffic while using Cross Connect: As the controller uses EndpointSlices to connect services, it may lead to looping traffic. However, the probability of traffic being stuck in a loop is low.
• Loss of connections: There is no fallback mechanism to restore connections if the migration fails. The controller assumes that the target cluster is ready to accept traffic before starting the migration.

TBA

• Object support: Support for more Kubernetes objects like StatefulSets, DaemonSets, etc.
• Secret and ConfigMap Migration: Ability to migrate secrets and config maps along with deployments
• Volume Migration: Support for migrating persistent volumes
• Service Connection Methods: More methods like DNS-based routing, etc.

Getting Started

Prerequisites

• Go
• Docker
• kubectl
• Access to two Kubernetes (source and target) on kind

KUBECONFIG

Ensure the kubeconfig to the target cluster is available at ./config/kc. Verify that the server ip to the target API server is reachable from the source cluster. The controller will use this kubeconfig to connect to the target cluster.

Installation

Run the installation using

make install

and build the controller image using

make docker-kind

This creates a Docker image for the controller that can be used in a Kubernetes cluster, and adds it to the local kind cluster.

Deploying the Controller

To deploy the MigrationController, run:

make deploy

Example Migration Configuration

Example yaml can be found in config/samples/batch_v1_migrationcontroller.yaml.

An example deployment can be created in the source cluster using:

make reset-deploy

Apply the crd to the source cluster:

kubectl apply -f config/samples/batch_v1_migrationcontroller.yaml
#OR
make deploy-cr

This should spawn 10 pods in the source cluster and create a service that connects to them.

The crd will create a MigrationController resource that manages the migration process.

Pause and Rollback Features

Pausing Migrations

You can pause an ongoing migration by setting the paused field to true in the MigrationController spec:

spec:
  paused: true
  # other fields...

This will:

• Immediately pause the migration process
• Maintain the current state of deployments in both source and target clusters
• Allow you to resume the migration later by setting paused back to false

Pausing is useful when:

• You need to investigate issues during migration
• You want to perform system checks or tests in the middle of a migration
• You need to delay completion for operational reasons

Rolling Back Migrations

If you need to revert a migration (either completed or in-progress), set the rollback field to true:

spec:
  rollback: true
  # other fields...

The rollback process:

• For both strategies: Immediately sets the source cluster deployment to its initial replica count
• Waits for the source deployment to become healthy using the same probe type (readiness/liveness) as configured
• Only after the source is healthy, completely erases the deployment from the target cluster
• Blocks any further migration attempts until rollback status is reset
• Updates service connections to restore traffic to the source cluster

Key benefits of the rollback mechanism:

• Immediate recovery: Restores full capacity to source cluster without delay
• Safe transition: Ensures source deployment is healthy before removing the target deployment
• Complete cleanup: Ensures no resources are left in target cluster
• Safe state: Prevents accidental migration continuation after rollback

The rollback status is tracked in the MigrationController status under the fields rollbackInProgress and rollbackCompleted.

To reset the rollback status and enable a new rollback operation, set the rollback field back to false:

spec:
  rollback: false
  # other fields...

This will clear both the rollbackInProgress and rollbackCompleted status flags, allowing you to initiate another rollback if needed in the future.

Limitations

• The controller currently supports only deployments. Other resources like StatefulSets, DaemonSets, etc., are not yet supported.
• Proxy networking mode is not yet working, the service is not able to route traffic from the same cluster to the service endpoint.