Skip to content

test(helm): Add single-node kind cluster set-up script for chart validation. - #419

Open
20001020ycx wants to merge 1 commit into
y-scope:mainfrom
20001020ycx:test/2026-07-28-helm-setup-test
Open

test(helm): Add single-node kind cluster set-up script for chart validation.#419
20001020ycx wants to merge 1 commit into
y-scope:mainfrom
20001020ycx:test/2026-07-28-helm-setup-test

Conversation

@20001020ycx

@20001020ycx 20001020ycx commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Description

This PR adds a single-node kind cluster set-up flow for validating the Helm chart, mirroring the structure of CLP's tools/deployment/package-helm/ set-up scripts.

Note for reviewer:

  • The Spider services fail fast when their dependencies are unreachable (storage needs the database; scheduler/workers need storage), so pods go through a few CrashLoopBackOff restarts before converging. We have thoroughly discuss this behaviour offline and concludes this is a better behaviour rather the waitfor semantic implemented in CLP.
  • We will add more e2e testing on the deployed cluster within the testing scripts once test(e2e): Add neural-network integration test. #385 lands.

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

  1. ./tools/deployment/spider-helm/set-up-test.sh passed
  2. Re-ran the script against the existing cluster to confirm the delete-and-recreate path works.

Summary by CodeRabbit

  • New Features

    • Added a streamlined setup workflow for testing the Helm deployment in a local Kubernetes cluster.
    • Added support for supplying custom storage, scheduler, and worker images during test setup.
    • Added automated checks that wait for all deployed components to become ready.
  • Chores

    • Updated the Helm chart version to 0.1.5.
    • Improved packaging so test setup scripts are excluded from the chart.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@20001020ycx, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 38 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: dd0e6f2b-e659-437e-9108-e69cdf8ba9e7

📥 Commits

Reviewing files that changed from the base of the PR and between 73a0750 and d4c99f0.

📒 Files selected for processing (4)
  • tools/deployment/spider-helm/.helmignore
  • tools/deployment/spider-helm/.set-up-common.sh
  • tools/deployment/spider-helm/Chart.yaml
  • tools/deployment/spider-helm/set-up-test.sh

Walkthrough

Changes

Helm test cluster setup

Layer / File(s) Summary
Shared setup utilities
tools/deployment/spider-helm/.set-up-common.sh
Adds shared Bash helpers for parsing image arguments, preparing kind clusters, generating Helm image values, and polling pod readiness.
Test setup integration
tools/deployment/spider-helm/set-up-test.sh, tools/deployment/spider-helm/Chart.yaml, tools/deployment/spider-helm/.helmignore
Adds a single-node kind test setup, installs the chart with configured images, waits for ready pods, bumps the chart version, and excludes setup scripts from packaging.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant SetUpTest
  participant Kind
  participant Helm
  participant Kubectl
  User->>SetUpTest: provide image arguments
  SetUpTest->>Kind: create named cluster
  SetUpTest->>Helm: install chart with image values
  SetUpTest->>Kubectl: wait for pods to become Ready
Loading

Suggested reviewers: junhaoliao

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding a single-node Kind cluster setup script for Helm chart validation.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Fix failing CI checks
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@20001020ycx
20001020ycx force-pushed the test/2026-07-28-helm-setup-test branch 2 times, most recently from 5f09a5c to 73a0750 Compare July 29, 2026 15:20
@20001020ycx
20001020ycx marked this pull request as ready for review July 29, 2026 15:37
@20001020ycx
20001020ycx requested review from a team and sitaowang1998 as code owners July 29, 2026 15:37

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tools/deployment/spider-helm/set-up-test.sh`:
- Around line 28-31: The image loading currently occurs inside
get_image_helm_args command substitutions, allowing failures to be ignored
before helm install. In tools/deployment/spider-helm/.set-up-common.sh lines
36-37, keep get_image_helm_args limited to emitting Helm --set flags; in
tools/deployment/spider-helm/set-up-test.sh lines 28-31, add a checked
image-loading preflight before helm install and abort if any kind load
docker-image operation fails.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 391b122d-78b8-4e89-a506-fbdecb4ed286

📥 Commits

Reviewing files that changed from the base of the PR and between 2f33edd and 73a0750.

📒 Files selected for processing (4)
  • tools/deployment/spider-helm/.helmignore
  • tools/deployment/spider-helm/.set-up-common.sh
  • tools/deployment/spider-helm/Chart.yaml
  • tools/deployment/spider-helm/set-up-test.sh

Comment on lines +28 to +31
helm install "${RELEASE_NAME}" "${script_dir}" \
$(get_image_helm_args "${CLUSTER_NAME}" "storage" "${STORAGE_IMAGE}") \
$(get_image_helm_args "${CLUSTER_NAME}" "scheduler" "${SCHEDULER_IMAGE}") \
$(get_image_helm_args "${CLUSTER_NAME}" "worker" "${WORKER_IMAGE}")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== files =="
fd -a '(\.set-up-common\.sh|set-up-test\.sh)$' . | sed 's#^\./##'

echo "== target files line counts =="
wc -l tools/deployment/spider-helm/.set-up-common.sh tools/deployment/spider-helm/set-up-test.sh

echo "== relevant target files =="
cat -n tools/deployment/spider-helm/.set-up-common.sh
printf '\n--- set-up-test.sh ---\n'
cat -n tools/deployment/spider-helm/set-up-test.sh

echo "== helpers references =="
rg -n "get_image_helm_args|load docker-image|kind load|helm install|set -|set -e|get_image" tools/deployment/spider-helm/.set-up-common.sh tools/deployment/spider-helm/set-up-test.sh

Repository: y-scope/spider

Length of output: 7540


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== bash command-substitution exit-status probe =="
bash -c 'set -o errexit; set -o pipefail; f() { echo x >/dev/null; false >&2; echo y; }; echo "start"; x=$(f); echo "x=$x"; echo "outdone" || echo "cmd exited non-zero; final status=$?"'

echo "== check bash availability and version =="
command -v bash
bash --version | head -n1

Repository: y-scope/spider

Length of output: 338


Separate checked image loading from Helm argument generation. A failed kind load docker-image inside command substitution is not a script error and will not stop the surrounding helm install from running.

  • Move the image-load step into a checked preflight before helm install.
  • Keep get_image_helm_args() limited to emitting the Helm --set flags.
📍 Affects 2 files
  • tools/deployment/spider-helm/set-up-test.sh#L28-L31 (this comment)
  • tools/deployment/spider-helm/.set-up-common.sh#L36-L37
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tools/deployment/spider-helm/set-up-test.sh` around lines 28 - 31, The image
loading currently occurs inside get_image_helm_args command substitutions,
allowing failures to be ignored before helm install. In
tools/deployment/spider-helm/.set-up-common.sh lines 36-37, keep
get_image_helm_args limited to emitting Helm --set flags; in
tools/deployment/spider-helm/set-up-test.sh lines 28-31, add a checked
image-loading preflight before helm install and abort if any kind load
docker-image operation fails.

@20001020ycx
20001020ycx force-pushed the test/2026-07-28-helm-setup-test branch from 697b4a2 to d4c99f0 Compare July 29, 2026 15:58

@sitaowang1998 sitaowang1998 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also need a explicit cleanup script/task.

# @param {string} component Image component name (e.g., "storage", "scheduler", "worker")
# @param {string} [image] Docker image (e.g., "spider-worker:dev")
# @return Prints helm --set flags to stdout
get_image_helm_args() {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function name should describe that it loads the images into kind cluster.

#!/usr/bin/env bash

# Single-node kind cluster set-up for testing the Helm chart
# TODO: Submit a job through the deployed stack once an end-to-end test scenario is available.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we start the client submitting the job in the bash script? More generally, how much of work should we put in bash script, and how much should we put in taskfiles?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants