Skip to content

Conversation

amaltaro
Copy link
Contributor

@amaltaro amaltaro commented Sep 2, 2025

Fixes #12421

Status

not-tested

Description

With this change, we consider full container data placement for Release Validation workflows as well - making it consistent with the Dataset WorkQueue policy defined at the WMSpec factory.

Is it backward compatible (if not, which system it affects?)

YES (won't fix workflows already placed at block-level though)

Related PRs

None

External dependencies / deployment changes

None

@dmwm-bot
Copy link

dmwm-bot commented Sep 2, 2025

Jenkins results:

  • Python3 Unit tests: succeeded
    • 1 changes in unstable tests
  • Python3 Pylint check: succeeded
    • 1 warnings
    • 11 comments to review
  • Pycodestyle check: succeeded
    • 2 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/WMCore-PR-Report/974/artifact/artifacts/PullRequestReport.html

@dmwm-bot
Copy link

dmwm-bot commented Sep 2, 2025

Jenkins results:

  • Python3 Unit tests: succeeded
  • Python3 Pylint check: succeeded
    • 1 warnings
    • 11 comments to review
  • Pycodestyle check: succeeded
    • 2 comments to review

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/WMCore-PR-Report/975/artifact/artifacts/PullRequestReport.html

@amaltaro
Copy link
Contributor Author

amaltaro commented Sep 2, 2025

We can actually test this setup even without patching MSTransferor.
I modified the SC_Nano.json template to be considered a RelVal workflow, similar to TC_Nano.json - which is right now a SubRequestType: Redigi and injected them both into testbed. Workqueue start policy looks okay:

amaltaro_SC_Nano_July2025_Val_250902_200709_9788.policies.start.policyName = 'Dataset'
amaltaro_TC_Nano_July2025_Val_250902_200721_4566.policies.start.policyName = 'Dataset'

and MSTransferor behaved as expected (RelVal policy takes precedence over NanoAOD):

2025-09-02 20:09:48,326:INFO:RequestInfo: Processing request: amaltaro_SC_Nano_July2025_Val_250902_200709_9788, with transferor template: RelValWorkflow, ...
### vs
2025-09-02 20:09:48,326:INFO:RequestInfo: Processing request: amaltaro_TC_Nano_July2025_Val_250902_200721_4566, with transferor template: NanoWorkflow, ...

Now if we apply this patch to testbed - hence ensuring that NanoAOD policy takes precedence over RelVal one - we can see that both workflows are considered as NanoWorkflow in MSTransferor:

2025-09-02 21:19:30,051:INFO:RequestInfo: Processing request: amaltaro_SC_Nano_July2025_Val_250902_210955_4568, with transferor template: NanoWorkflow, ...
### vs
2025-09-02 21:19:30,052:INFO:RequestInfo: Processing request: amaltaro_TC_Nano_July2025_Val_250902_211012_1400, with transferor template: NanoWorkflow, ...

If this is the desired behavior for RelVal, then this patch is ready to go.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rucio rules created on block level instead of dataset level
2 participants