Add backup and restore commands with Postgres 18 upgrade

[jouwdan]

Summary

This PR introduces community backup and community restore commands that bundle database dumps and uploads together for consistent disaster recovery. It also upgrades the stack from Postgres 16 to Postgres 18 and adds the postgres client tools to the runtime image.

Key Changes

• New backup/restore system (apps/cli/src/backup.ts):

  • community backup command creates a single .tar.gz bundle containing a pg_dump, uploads (local or S3), and a manifest
  • community restore command restores bundles into a new, empty database with uploads
  • Handles both local disk and S3 uploads with sensible defaults (include local, skip S3 unless explicitly requested)
  • Uses DIRECT_DATABASE_URL for dumps to avoid transaction pooler issues
  • Includes comprehensive validation and user-friendly error messages

• S3FileStore enhancement (packages/drivers/src/files/s3-file-store.ts):

  • Added listKeys() async generator to paginate through all objects in a bucket
  • Enables pulling entire S3 buckets into backup bundles with --uploads include

• Postgres 18 upgrade:

  • Updated all compose files and CI workflows from postgres:16-alpine to postgres:18-alpine
  • Added postgresql18-client to the runtime Docker image so pg_dump, pg_restore, and psql are available
  • Documented migration path for existing deployments crossing major versions

• CLI integration (apps/cli/src/index.ts):

  • Registered backup and restore commands with appropriate usage strings

• Comprehensive testing:

  • Unit tests for manifest parsing, bundle naming, content type detection, and byte formatting
  • Full round-trip integration test in CI that seeds a board, backs it up, restores to a fresh database, and verifies content survival
  • Tests verify refusal to restore over non-empty databases

• Documentation updates:

  • Updated operating guide with backup/restore procedures and rationale
  • Added upgrading guide section for Postgres major version transitions
  • Updated disaster recovery procedures to use new commands
  • Clarified that backups are operator responsibility, not panel-only

Notable Implementation Details

• Backup manifests are JSON with format versioning to support future changes
• Bundles use standard tar.gz and pg_dump --format=custom for portability and selective restore
• Uploads are included by default for local deployments (where they live on the machine) but skipped for S3 (where they're already off-machine)
• The restore command validates that target databases are empty to prevent accidental overwrites
• Post-restore migrations are applied automatically and reported to the user
• All postgres client tools are in the image so the commands work everywhere the CLI works

https://claude.ai/code/session_01LYancK1YxtRYHHEmtoqRPA