Skip to content

fix(konflux-build-triage): fix dispatcher dedup, filtering, and throttling - #29

Open
quay-devel wants to merge 1 commit into
mainfrom
fix/konflux-build-triage-dispatcher
Open

fix(konflux-build-triage): fix dispatcher dedup, filtering, and throttling#29
quay-devel wants to merge 1 commit into
mainfrom
fix/konflux-build-triage-dispatcher

Conversation

@quay-devel

Copy link
Copy Markdown
Contributor

Summary

Five shortcomings found after the first live triage run are addressed here:

  • Dedup was broken: session_name is ignored by ACP (generates UUID names). Switched to displayName-based matching with acp_list_sessions(search="Fix:") — both dedup and triage-cap checks now key off displayName.
  • EOL versions triaged: Added SUPPORTED_VERSIONS allowlist and MAX_FAILURE_AGE_DAYS=30 env var. A new Step 2.5 filters failures before spawning, with per-entry logging.
  • FBC empty-branch components: check-build-health.sh now skips and warns on components with empty branch in --failed-only mode. CLAUDE.md adds a branch filter as the first pre-spawn check.
  • No throttling: Added MAX_SESSIONS_PER_RUN=5 cap. Failures are sorted by last_build recency before the loop so the cap favors recent breakage.
  • Stuck sessions: Added post-spawn session health check guidance — log idle session IDs (totalMessages=0) in the report for investigation without blocking the dispatcher.

Files changed

File Change
workflows/konflux-build-triage/CLAUDE.md Rewrote dedup section, added Step 2.5 filter, updated Step 3 (sort + run cap), updated report format, added stuck-session error note
workflows/konflux-build-triage/.ambient/ambient.json Added MAX_SESSIONS_PER_RUN, SUPPORTED_VERSIONS, MAX_FAILURE_AGE_DAYS env vars
workflows/konflux-build-triage/scripts/check-build-health.sh Skip + warn on empty-branch components in --failed-only JSON path

Test plan

  • Trigger a triage run and verify dedup works across cron boundaries (no re-spawn of already-triaged pipelineruns)
  • Set SUPPORTED_VERSIONS=v3-18 and confirm only matching applications are spawned
  • Confirm FBC components no longer produce empty-branch sessions
  • Verify run cap at 5 sessions with a note about deferred failures in the report
  • Check that oldest failures are skipped first when cap is hit

🤖 Generated with Claude Code

…tling

Five shortcomings identified after the first live triage run:

1. Deduplication used session_name which ACP ignores (generates UUID names).
   Switch to displayName-based matching via acp_list_sessions(search="Fix:").

2. Stale/EOL versions were triaged unnecessarily. Add SUPPORTED_VERSIONS
   allowlist and MAX_FAILURE_AGE_DAYS (default 30) to filter them out.

3. FBC components with empty branch fields caused sessions to default to
   main and fail immediately. check-build-health.sh now skips empty-branch
   components with a warning in --failed-only mode; CLAUDE.md adds a branch
   filter step before spawning.

4. 11 sessions spawned simultaneously with no prioritization. Add
   MAX_SESSIONS_PER_RUN cap (default 5) and sort failures by last_build
   recency so the most recent breakage gets triaged first.

5. Spawned sessions appeared stuck (totalMessages=0). Add post-spawn
   session health check guidance and log idle session IDs for investigation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented May 12, 2026

Copy link
Copy Markdown

Walkthrough

Configuration and documentation for the Konflux build triage dispatcher are updated with environment-driven limits (MAX_SESSIONS_PER_RUN, MAX_FAILURE_AGE_DAYS, SUPPORTED_VERSIONS), new failure filtering and deduplication logic, and a corresponding script change to implement branch-emptiness skipping in health checks.

Changes

Build Triage Workflow Configuration and Filtering

Layer / File(s) Summary
Environment configuration for limits and filters
workflows/konflux-build-triage/.ambient/ambient.json
New environment variables define per-run session cap, version allowlist, and failure age cutoff thresholds that control triage behavior.
Failure filtering and deduplication strategy
workflows/konflux-build-triage/CLAUDE.md (lines 40–68, 114–163)
Workflow introduces multi-stage failure filtering: skip components with empty branches, optionally filter by supported versions, and apply age-based cutoff. Deduplication now matches ACP session displayName against a deterministic "Fix: {component} ({pipelinerun})" format, with active-session counting for triage-cap enforcement.
Run cap and session spawning
workflows/konflux-build-triage/CLAUDE.md (lines 23–26, 178–189)
Per-run cap skips failures before dedup/triage-cap checks; session creation uses arbitrary slug for session_name while display_name remains deterministic for dedup matching.
Empty-branch filtering in health check
workflows/konflux-build-triage/scripts/check-build-health.sh
When --failed-only mode is enabled, components with both build_failed==true and empty branch are skipped from JSON output with a warning logged to stderr.
Status verification and expanded reporting
workflows/konflux-build-triage/CLAUDE.md (lines 250–254, 198–211)
Post-spawn logic optionally calls status checks on spawned sessions to detect idle startup; run summary expands to include filtered/version/age/branch counts and separate run-cap vs triage-cap skip breakdowns.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/konflux-build-triage-dispatcher

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@workflows/konflux-build-triage/CLAUDE.md`:
- Around line 120-122: Add a language identifier to the Markdown fenced code
blocks flagged by markdownlint (MD040) so they are not treated as unspecified
fences; specifically update the fences containing the strings "SKIP {component}
— empty branch (FBC or unresolved source ref)", "SKIP {component} (app:
{application}) — not in SUPPORTED_VERSIONS ({SUPPORTED_VERSIONS})", "SKIP
{component} — last build {last_build} older than {MAX_FAILURE_AGE_DAYS} days",
and "NOTE: Run cap reached. {C} failure(s) deferred to next run." by changing
their opening ``` to ```text (or another appropriate language), ensuring each
matching closing ``` remains unchanged.

In `@workflows/konflux-build-triage/scripts/check-build-health.sh`:
- Around line 215-218: The --failed-only empty-branch filter is only applied in
the JSON branch; update check-build-health.sh so components with an empty branch
are excluded whenever FAILED_ONLY=true and build_failed=true regardless of
output mode—either move the if check (using variables FAILED_ONLY, build_failed,
branch, comp_name, app) to run before the output-mode branching, or duplicate
the same condition into the table-mode flow so FBC entries with empty branch are
skipped for all --failed-only outputs.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: 71e9d4f3-7362-4b36-827a-3e0080b3e1c9

📥 Commits

Reviewing files that changed from the base of the PR and between a8feb25 and dafdb72.

📒 Files selected for processing (3)
  • workflows/konflux-build-triage/.ambient/ambient.json
  • workflows/konflux-build-triage/CLAUDE.md
  • workflows/konflux-build-triage/scripts/check-build-health.sh
📜 Review details
🧰 Additional context used
📓 Path-based instructions (1)
workflows/konflux-build-triage/scripts/check-build-health.sh

📄 CodeRabbit inference engine (workflows/konflux-build-triage/CLAUDE.md)

Use bash script 'scripts/check-build-health.sh --failed-only' to assess build health via KubeArchive REST API

In check-build-health.sh: exclude components with empty 'branch' fields (FBC components) from --failed-only output

Use KubeArchive REST API with curl and bearer token auth to query latest on-push PipelineRuns per component

Files:

  • workflows/konflux-build-triage/scripts/check-build-health.sh
🪛 markdownlint-cli2 (0.22.1)
workflows/konflux-build-triage/CLAUDE.md

[warning] 120-120: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


[warning] 127-127: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


[warning] 134-134: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


[warning] 208-208: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🔇 Additional comments (2)
workflows/konflux-build-triage/.ambient/ambient.json (1)

10-13: Env var additions look consistent with dispatcher behavior changes.

The new runtime knobs align with the documented filtering/throttling flow and have sane defaults.

workflows/konflux-build-triage/CLAUDE.md (1)

44-69: Dispatcher flow updates are clear and operationally stronger.

The displayName-based dedup plus pre-spawn filtering and run-cap ordering materially improve determinism and reduce noisy triage.

Also applies to: 114-164, 198-210

Comment on lines +120 to +122
```
SKIP {component} — empty branch (FBC or unresolved source ref)
```

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Add languages to fenced code blocks to satisfy markdownlint (MD040).

These new fences are missing a language identifier, which is currently flagged by static analysis.

Suggested fix
-```
+```text
 SKIP {component} — empty branch (FBC or unresolved source ref)

- +text
SKIP {component} (app: {application}) — not in SUPPORTED_VERSIONS ({SUPPORTED_VERSIONS})


-```
+```text
SKIP {component} — last build {last_build} older than {MAX_FAILURE_AGE_DAYS} days

- +text
NOTE: Run cap reached. {C} failure(s) deferred to next run.

Also applies to: 127-129, 134-136, 208-210

🧰 Tools
🪛 markdownlint-cli2 (0.22.1)

[warning] 120-120: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@workflows/konflux-build-triage/CLAUDE.md` around lines 120 - 122, Add a
language identifier to the Markdown fenced code blocks flagged by markdownlint
(MD040) so they are not treated as unspecified fences; specifically update the
fences containing the strings "SKIP {component} — empty branch (FBC or
unresolved source ref)", "SKIP {component} (app: {application}) — not in
SUPPORTED_VERSIONS ({SUPPORTED_VERSIONS})", "SKIP {component} — last build
{last_build} older than {MAX_FAILURE_AGE_DAYS} days", and "NOTE: Run cap
reached. {C} failure(s) deferred to next run." by changing their opening ``` to
```text (or another appropriate language), ensuring each matching closing ```
remains unchanged.

Comment on lines +215 to +218
if [[ "$FAILED_ONLY" == "true" && "$build_failed" == "true" && -z "$branch" ]]; then
echo "WARNING: Skipping ${comp_name} (app: ${app}) — empty branch, cannot spawn fix session" >&2
continue
fi

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Empty-branch filter is only applied in JSON mode, not all --failed-only output.

This condition is scoped to the JSON branch, so --failed-only --table still emits FBC entries with empty branch. Please apply the same skip in table mode (or centralize filtering before output-mode branching).

Suggested fix
@@
   while IFS=$'\t' read -r comp_name app source branch; do
@@
     if [[ "$FAILED_ONLY" == "true" && "$build_failed" != "true" ]]; then
       continue
     fi
+
+    if [[ "$FAILED_ONLY" == "true" && "$build_failed" == "true" && -z "$branch" ]]; then
+      echo "WARNING: Skipping ${comp_name} (app: ${app}) — empty branch, cannot spawn fix session" >&2
+      continue
+    fi
@@
   done < "$TMPDIR/components.tsv" | sort -t$'\t' -k1,1
As per coding guidelines, "In check-build-health.sh: exclude components with empty 'branch' fields (FBC components) from --failed-only output".
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@workflows/konflux-build-triage/scripts/check-build-health.sh` around lines
215 - 218, The --failed-only empty-branch filter is only applied in the JSON
branch; update check-build-health.sh so components with an empty branch are
excluded whenever FAILED_ONLY=true and build_failed=true regardless of output
mode—either move the if check (using variables FAILED_ONLY, build_failed,
branch, comp_name, app) to run before the output-mode branching, or duplicate
the same condition into the table-mode flow so FBC entries with empty branch are
skipped for all --failed-only outputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant