Skip to content

SWF v42 baseline: Find Data with LLM dialog, composed-name integrity, GPU worker, notice routing - #70

Merged
wenaus merged 6 commits into
mainfrom
infra/baseline-v42
Aug 15, 2026
Merged

SWF v42 baseline: Find Data with LLM dialog, composed-name integrity, GPU worker, notice routing#70
wenaus merged 6 commits into
mainfrom
infra/baseline-v42

Conversation

@wenaus

@wenaus wenaus commented Aug 15, 2026

Copy link
Copy Markdown
Member

Release v42 of the coordinated SWF baseline. Release notes for the cycle are in RELEASE_NOTES.md.

Highlights across the triple and its peer repositories:

  • Find Data and the Brains dialog — one search field over the recorded dataset inventory with a chat over the search on the DISpatcher bot engine.
  • PCS composed-name integrity — discriminating identity at intake, a resolver that refuses ambiguity, catalog backfill, and an integrity collector.
  • PanDA task operations — retry coverage for aborted tasks, dataset reopen, sandbox and log-dataset keepalive.
  • Notice routing — one subscription-driven router replaces direct posting.
  • GPU worker outside the SCDF perimeter — BNL_NPPS_GPU with a standalone pilot and object-store stage-out.
  • Data provenance, validation interface v1, residual rerun.

swf-testbed in this cycle: the BNL_NPPS_GPU queue definition, testbed run --notice, CLI robustness fixes, retired diagrams superseded by their epic-wfms-docs versions, and the release notes.

🤖 Generated with Claude Code

wenaus and others added 6 commits August 8, 2026 19:17
_check_tcp_service() warns instead of aborting when a port env var is
not an integer; drop an unused import.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The flag travels as a command parameter to the workflow runner and
lands top-level in the execution's parameter_values, where the
monitor's terminal-transition event carries it to notice routing
(swf-monitor docs/NOTICE_ROUTING.md). Only stamped runs emit
notice=true; the nightly cron is the intended user.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove E0-E1_workflow_schematic.svg (and its png export),
idds-panda-detail-v1.svg, diagram5_panda_provenance_epic.svg, and
agent-management-overview-v4.svg; the maintained versions live in
epic-wfms-docs docs/diagrams/. Doc references now point at the
epic-wfms-docs copies.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cloned from NERSC_Perlmutter_epic_gpu_test with npps0 values from
today's box survey: corecount 16 and maxrss 48000 (40 cores, 125 GB,
two GPU job slots), maxwdir and maxinputsize 30000 (no scratch volume,
/home 97% full), is_cvmfs false (no CVMFS; container by registry
pull), container_options --nv, NERSC binds and environ dropped.
Authoring field set only; CRIC-computed fields absent.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The facility was renamed; DNS still carries sdcc.bnl.gov, so hostnames
keep the old string while prose uses the current name. The quoted source
material in e0-e1-interface-source-notes.md stays verbatim.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings August 15, 2026 11:28
@wenaus
wenaus merged commit 27981a6 into main Aug 15, 2026
3 checks passed

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Release-baseline v42 update for swf-testbed, primarily adding a testbed run --notice stamp that propagates through orchestrator → workflow-runner so workflow executions can emit subscription-routed completion notices, plus baseline config/docs updates (GPU queue definition, diagram retirements, release notes).

Changes:

  • Add --notice to testbed run and propagate it as a top-level execution attribute (parameter_values.notice) via orchestrator and workflow-runner.
  • Add the BNL_NPPS_GPU PanDA queue definition to the testbed queue catalog.
  • Update documentation: retire local diagram assets in favor of epic-wfms-docs hosted diagrams; minor terminology corrections; add v42 release notes.

Reviewed changes

Copilot reviewed 8 out of 13 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
workflows/workflow_runner.py Promotes notice from command parameter to a top-level stored execution attribute.
workflows/orchestrator.py Threads notice through config → run parameters and exposes it on orchestrator run(...).
src/swf_testbed_cli/main.py Adds testbed run --notice option and improves TCP probe robustness.
RELEASE_NOTES.md Adds v42 release notes content.
docs/images/idds-panda-detail-v1.svg Removes a retired diagram asset.
docs/images/agent-management-overview-v4.svg Removes a retired diagram asset.
docs/e0-e1-interface.md Updates SDCC→SCDF terminology.
docs/E0-E1_workflow_schematic.svg Removes a retired diagram asset.
docs/diagram5_panda_provenance_epic.svg Removes a retired diagram asset.
docs/architecture.md Switches schematic link to epic-wfms-docs hosted diagram.
docs/agent-management.md Switches overview diagram link to epic-wfms-docs hosted diagram.
config/panda_queues.json Adds the BNL_NPPS_GPU queue definition.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines 356 to 370
@app.command()
def run(
config_name: str = typer.Argument(
None,
help="Config name (e.g., 'fast_processing' loads workflows/fast_processing.toml). "
"If not specified, uses workflows/testbed.toml"
),
notice: bool = typer.Option(
False, "--notice",
help="Stamp this run for a completion notice: its terminal state "
"emits a workflow_execution_completed event with notice=true, "
"deliverable to notice-routing subscribers "
"(swf-monitor docs/NOTICE_ROUTING.md)."
)
):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants