Skip to content

chore: couch db version upgrade guide and instrumentation#2204

Open
PeterBaker0 wants to merge 7 commits into
mainfrom
feat/recovery-checks
Open

chore: couch db version upgrade guide and instrumentation#2204
PeterBaker0 wants to merge 7 commits into
mainfrom
feat/recovery-checks

Conversation

@PeterBaker0

Copy link
Copy Markdown
Contributor

Summary

  • Add an AWS CouchDB version-upgrade runbook plus helper scripts (baseline, EBS snapshot, stop/detach for replace).
  • Force Couch EC2 replacement on UserData changes (userDataCausesReplacement) and document the post-replace keys/migrate flow. NOTE this will force an instance replacement at deploy time even if you don't change the version.
  • Add dump-user-emails for maintenance notices; have env-from-cdk-stack.sh emit AWS_DEFAULT_REGION.

Tested out on our stage replica. Works well. Note in the above that if you don't have the userDataCausesReplacement you don't get an EC2 replacement with a new user data run (to build the new couch version) but it does mean for current builds you'll get an instance replacement which is a bit risky if not coordinated well. Future ones won't experience that unless the user data hash changes (which it shouldn't typically).

Note if you don't detach the volume prior to running the instance replacement, you'll get a CFN error about the volume being already attached. Known CFN bug we have to work around.

…rd name

Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
@PeterBaker0
PeterBaker0 requested a review from stevecassidy July 23, 2026 03:56
@github-actions

github-actions Bot commented Jul 23, 2026

Copy link
Copy Markdown

E2E ✅ passed

Result ✅ Suite passed
Artifacts Download e2e-artifacts-29983214908.zip
Run View workflow run

After downloading and extracting the .zip file, open artifacts/index.html (failures listed first).

Updated for commit bd82efe

Signed-off-by: Peter Baker <peter.baker122@csiro.au>
@PeterBaker0 PeterBaker0 changed the title feat chore: couch db version upgrade guide and instrumentation Jul 23, 2026
Signed-off-by: Peter Baker <peter.baker122@csiro.au>
Signed-off-by: Peter Baker <peter.baker122@csiro.au>

@stevecassidy stevecassidy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ok with a few comments.

@@ -0,0 +1,278 @@
#!/usr/bin/env bash

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be in the AWS folder please.

@@ -0,0 +1,189 @@
/* eslint-disable n/no-process-exit */

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This facility would be useful but better as an API endpoint for SYSTEM_ADMIN so that we can easily access it on a given server. Maybe a separate PR.

@@ -0,0 +1,77 @@
{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this supposed to be here?

@@ -0,0 +1,198 @@
# CouchDB version upgrade on AWS

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is AWS specific but the 'migrate-with-keys' step after updating seems general, might be worth pointing that out near the top along with any other general issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants