Skip to content

feat: aggregate group readiness in cluster status - #7023

Merged
ti-chi-bot[bot] merged 5 commits into
pingcap:mainfrom
csuzhangxc:cluster-synced
Aug 10, 2026
Merged

feat: aggregate group readiness in cluster status#7023
ti-chi-bot[bot] merged 5 commits into
pingcap:mainfrom
csuzhangxc:cluster-synced

Conversation

@csuzhangxc

@csuzhangxc csuzhangxc commented Aug 7, 2026

Copy link
Copy Markdown
Member

What

  • add Cluster-level Ready and Synced conditions
  • aggregate each condition independently across all observed Group types
  • require every Group condition to be True for the current Group and Cluster generations
  • keep the existing Suspended aggregation on the shared condition helper
  • report Ready=False and Synced=False when no Groups are observed

Why

Suspended=False only indicates that the desired state has switched to resume. It does not guarantee that all compute Groups have finished reconciling and become usable.

With these Cluster-level conditions, an upper-level controller can determine resume completion by reading only the Cluster:

  • effective suspendCompute == false
  • current-generation Suspended=False
  • current-generation Ready=True
  • current-generation Synced=True

Ready and Synced are aggregated independently, so they also show whether resume is waiting for readiness or synchronization.

Notes

The Cluster API does not own the desired Group topology. The aggregation therefore covers Groups currently observed by the controller cache. Callers must ensure that the Group topology is complete and stable while using these conditions as pause/resume completion signals.

For now, observedClusterGeneration is validated by the Cluster aggregation. Binding it directly to Group/Instance Ready and Synced updates is left as a follow-up.

Tests

go test ./pkg/controllers/cluster/...

Summary by CodeRabbit

  • New Features

    • Added clear cluster status indicators for readiness and synchronization.
    • Cluster readiness and synchronization now reflect the status of all observed groups.
    • Conditions remain false when groups are missing, outdated, or not ready.
    • Status messages identify groups preventing the cluster from becoming ready or synchronized.
  • Bug Fixes

    • Improved cluster status accuracy when group conditions change, become stale, or are removed.

@ti-chi-bot
ti-chi-bot Bot requested a review from shonge August 7, 2026 06:56
@github-actions github-actions Bot added the v2 for operator v2 label Aug 7, 2026
@ti-chi-bot ti-chi-bot Bot added the size/L label Aug 7, 2026
@coderabbitai

coderabbitai Bot commented Aug 7, 2026

Copy link
Copy Markdown

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 344cd260-161a-4297-ae8f-cf62fccdbd07

📥 Commits

Reviewing files that changed from the base of the PR and between c0c6c2a and f61232a.

📒 Files selected for processing (1)
  • pkg/controllers/cluster/tasks/status.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/controllers/cluster/tasks/status.go

📝 Walkthrough

Walkthrough

The cluster API now defines Ready and Synced condition constants. Cluster status aggregation evaluates these conditions across observed Groups and the current Cluster generation. Tests cover true, false, stale, and no-Group states.

Changes

Cluster condition aggregation

Layer / File(s) Summary
Condition contract and aggregation
api/core/v1alpha1/cluster_types.go, pkg/controllers/cluster/tasks/status.go
Added condition constants and generic helpers. The status updater aggregates suspension, readiness, and synchronization across Group collections.
Condition aggregation validation
pkg/controllers/cluster/tasks/status_test.go
Updated initial condition expectations and added coverage for current-generation, stale, false, and no-Group cases.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ClusterStatusUpdater
  participant GroupStatuses
  participant ConditionHelpers
  participant ClusterStatus
  ClusterStatusUpdater->>GroupStatuses: Read observed Group conditions
  ClusterStatusUpdater->>ConditionHelpers: Aggregate Ready and Synced states
  ConditionHelpers-->>ClusterStatusUpdater: Return status, reason, and message
  ClusterStatusUpdater->>ClusterStatus: Set cluster conditions
Loading

Suggested reviewers: shonge

Poem

A rabbit checks the Groups at dawn,
Finds Ready and Synced written on.
Stale states fade, false flags show,
No Groups keep both conditions low.
The cluster status hops along.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: aggregating Group readiness in Cluster status.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codecov-commenter

codecov-commenter commented Aug 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 40.94%. Comparing base (40fd4dc) to head (f61232a).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #7023      +/-   ##
==========================================
+ Coverage   40.84%   40.94%   +0.09%     
==========================================
  Files         439      439              
  Lines       25472    25515      +43     
==========================================
+ Hits        10403    10446      +43     
  Misses      15069    15069              
Flag Coverage Δ
unittest 40.94% <100.00%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Cluster-level Ready and Synced conditions so higher-level controllers can determine pause/resume completion from the Cluster status by aggregating readiness/sync state across all observed Group types.

Changes:

  • Aggregate Group Ready and Synced into new Cluster conditions (ClusterCondReady, ClusterCondSynced) based on current Group + Cluster generations.
  • Refactor the per-Group-type condition checks into a generic helper (groupsConditionsTrue + allGroupConditionsTrue).
  • Extend status task tests to cover the new Cluster conditions and the “no observed Groups” case.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
pkg/controllers/cluster/tasks/status.go Adds Cluster Ready/Synced aggregation logic and refactors Group condition evaluation helpers.
pkg/controllers/cluster/tasks/status_test.go Updates baseline Cluster condition expectations and adds readiness aggregation test coverage.
api/core/v1alpha1/cluster_types.go Introduces new Cluster condition type constants: Ready and Synced.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pkg/controllers/cluster/tasks/status.go
Comment thread pkg/controllers/cluster/tasks/status.go
@csuzhangxc

Copy link
Copy Markdown
Member Author

/retest

@csuzhangxc

Copy link
Copy Markdown
Member Author

@csuzhangxc: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-e2e 8c52103 link true /test pull-e2e
Full PR test history. Your PR dashboard.

Details

Investigated the failed pull-e2e run. The two scale-out pods were created, but no old PD pods were deleted/recreated within 10 minutes, causing the watcher to time out. I found no direct evidence that PR #7023 caused this; it looks more like a timing/flakiness issue in the PDGroup rolling-update path.

Comment thread api/core/v1alpha1/cluster_types.go Outdated
ClusterCondSuspended = "Suspended"
ClusterSuspendReason = "ClusterSuspend"

ClusterCondReady = "Ready"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use existing CondReady

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread api/core/v1alpha1/cluster_types.go Outdated

ClusterCondReady = "Ready"
ClusterReadyReason = "ClusterReady"
ClusterCondSynced = "Synced"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use existing CondSynced

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread pkg/controllers/cluster/tasks/status.go Outdated
},
} {
status := metav1.ConditionFalse
message := fmt.Sprintf("Not all Groups have %s=True for the current Cluster generation", condition.conditionType)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add cond=false groups into msg?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@csuzhangxc

Copy link
Copy Markdown
Member Author

/retest

@liubog2008

Copy link
Copy Markdown
Member

/lgtm

@ti-chi-bot ti-chi-bot Bot added the lgtm label Aug 10, 2026
@ti-chi-bot

ti-chi-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liubog2008

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot

ti-chi-bot Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

[LGTM Timeline notifier]

Timeline:

  • 2026-08-10 02:06:21.545237291 +0000 UTC m=+3012167.581332357: ☑️ agreed by liubog2008.

@ti-chi-bot ti-chi-bot Bot added the approved label Aug 10, 2026
@ti-chi-bot
ti-chi-bot Bot merged commit 2ecbfb0 into pingcap:main Aug 10, 2026
11 checks passed
@csuzhangxc
csuzhangxc deleted the cluster-synced branch August 10, 2026 02:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants