Design : Controller

WIP

The Controller component is responsible for tracking which tables are being replicated, as well as handling the process behind adding and removing replications. It consists of a DynamoDB table and six lambda functions.

Diagram

Component Logic

The controller DynamoDB table holds entries for each replication, identified by the name of the replicated table. entry by a row in the controller DynamoDB table, identified by the name of the replicated table.

Steps:

• VALIDATE_SOURCE
• VALIDATE_REPLICA
• CREATE_REPLICA
• START_REPLICATION
• REPLICATING *
• STOP_REPLICATION

States:

• PENDING
• IN_PROGRESS
• COMPLETE
• FAILED
• ACTIVE *

Functions

Controller     -     [src]

The controller function will be invoked whenever an item is added, removed, or modified in the controller DynamoDB table, and acts depending on the values of the step and state fields in the new image.

Validate Source     -     [src]

The validate source function checks to see if the source table is configured properly for replication.
To be considered valid, the source table match must:

• Exist
• Have streams enabled
• Have a stream view type of NEW_AND_OLD_IMAGES

If any of these conditions are not met, the function will return unsuccessful, failing the replication process.

Validate Replica     -     [src]

The validate replica function checks to see if the replica table already exists. If the table exists, it compares the source and replica key schemas.

If the replica table exists and the key schemas do not match, the function returns unsuccessful, failing the replication process. Otherwise, the function returns successful.

If the table exists with a matching schema, the controller will skip the CREATE_REPLICA step and proceed directly to START_REPLICATION.

Create Replica     -     [src]

The create replica function creates a new DynamoDB table in the replica region to match the source.
The replica table will have the same name, key schema, attribute definitions, provisioned throughput, stream specification, local secondary indexes, and global secondary indexes as the source table.

It will return successful unless there was a issue creating the table.

Note: The create replica function does not copy table data

Start Replication     -     [src]

The start replication adds an event source mapping for the source table to the Replicator function, allowing it to start reading from the stream and writing to the replica table. It will return successfully unless there was an issue adding the event source mapping.

Note: The replicator starts reading data from the oldest point in the stream (up to 24 hours), so data processed during the setup will not be missed.

Stop Replication     -     [src]

The stop replication step removes the event source mapping for the source table from the replicator function, stopping the replication of data. It will return successfully unless there was an issue removing the event source mapping.