Skip to content

Conversation

jitendra1411
Copy link

Overview

This PR introduces a new Backup plugin architecture to the ClickHouse Operator, enabling automated backup operations via an external plugin service. Issue: #1795


Current PR Includes

1. Operator Enhancements

  • CRDs

    • ClickhouseBackup (CHB) — manage one-time backup requests with S3 configuration, whitelist/blacklist filtering, and method/plugin settings.
  • Controller Updates

    • Operator reconciles CHB/CHSB CRs and communicates with a gRPC backup plugin using JSON-serialized request data.
    • Adds plugin discovery via CR labels/annotations.
    • Implements lifecycle management and status reporting.

2. Go Plugin Skeleton

  • Introduces a gRPC interface for backup operations using BackupRequest and BackupResult messages.
  • Plugin architecture allows future extension for operations like restore, encryption, incremental backups, etc.

Motivation & Benefits

  • Modular Design: Backup behavior is decoupled from the operator — update, replace, or extend the plugin without touching core operator logic.
  • Extensibility: Support for diverse backup implementations (e.g., S3, local disk, incremental, etc.).
  • Production-Ready: Enables defined workflows for both one-off and scheduled backups, ensuring reliable defense against data loss.

TODO

  • Restore Workflow: Implement a corresponding restore service via plugin.
  • Test Cases: Implement the test cases
  • ClickhouseScheduledBackup (CHSB) Implement code for scheduled backup
  • Generic gRPC Client: Implement a reusable gRPC client inside the operator for communication with different plugins.

Testing Notes

  • Validate CHB and CHSB workflows manually using a mock plugin server.
  • Status updates should reflect backup success or failure states.
  • Scheduling logic (cron parsing, suspension, etc.) requires validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant