[iarspider]

No description provided.

[cmsbuild]

A new Pull Request was created by @iarspider for branch master.

@akritkbehera, @cmsbuild, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.
cms-bot commands are listed here

[cmsbuild]

cms-bot internal usage

[cmsbuild]

Pull request #2891 was updated.

[smuzaffar]

thanks @iarspider for looking in to this. How this change will behave for existing PRs processed already processby v2? I hope this change will change re-write the bot cache ... right?

[iarspider]

thanks @iarspider for looking in to this. How this change will behave for existing PRs processed already processby v2? I hope this change will change re-write the bot cache ... right?

Yes - unless the bot stops early (e.g. if it encounters commits from the future).

[smuzaffar]

Is it expected that new v2 will always go back to get files e.g. https://cmssdt.cern.ch/jenkins/job/cms-bot/2181120/console I see that bot is again calling get files

Requester.py:586 [PyGithub] GET https://api.github.com/repos/cms-data/MagneticField-Interpolation/commits/bc5739c26a9d97dfd664e433811c7c23cc3c2f4c
process_pr_v2.py:7169 [DEBUG] Fetched head commit bc5739c2
process_pr_v2.py:1197 [DEBUG] Successfully loaded cache from comments
process_pr_v2.py:7243 [DEBUG] PR is in draft state, disabling @-mentions
Requester.py:586 [PyGithub] GET https://api.github.com/repos/cms-data/MagneticField-Interpolation/pulls/6/files?per_page=100
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=2
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=3
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=4
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=5
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=6
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=7
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=8
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=9
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=10
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=11
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=12
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=13
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=14
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=15
Requester.py:586 [PyGithub] GET https://api.github.com/repositories/23923715/pulls/6/files?per_page=100&page=16

I thought that is base sha and PR sha are not changed then bot will re-use the previously cached files info

[cmsbuild]

Pull request #2891 was updated.

[smuzaffar]

also looks like every time bot process a PR then its cache comment is edited ( I guess when comment is compressed ), see cms-data/MagneticField-Interpolation#6 where comments are edited 3 times . I guess if you compare the uncompressed comment contents then you might be able to avoid the editting

[iarspider]

Checking. It should reuse existing cache if it exists.

[cmsbuild]

Pull request #2891 was updated.

[iarspider]

Is it expected that new v2 will always go back to get files e.g. https://cmssdt.cern.ch/jenkins/job/cms-bot/2181120/console I see that bot is again calling get files
(...)
I thought that is base sha and PR sha are not changed then bot will re-use the previously cached files info

I can't reproduce the problem with this branch - see https://cmssdt.cern.ch/jenkins/job/cms-bot/2181343/console:

process_pr_v2.py:7353 [DEBUG] Fetched head commit bc5739c2
process_pr_v2.py:1332 [DEBUG] Successfully loaded cache from comments
process_pr_v2.py:7427 [DEBUG] PR is in draft state, disabling @-mentions
process_pr_v2.py:4663 [DEBUG] PR files unchanged since bc5739c2, using cached file list

It also doesn't overwrite the cache:

process_pr_v2.py:1398 [DEBUG] Cache comment 1/3 unchanged, skipping
process_pr_v2.py:1398 [DEBUG] Cache comment 2/3 unchanged, skipping
process_pr_v2.py:1398 [DEBUG] Cache comment 3/3 unchanged, skipping

[smuzaffar]

I opened new cms-data/MagneticField-Interpolation#7 which was processed by new v2 bot but then a new bot job started with v1 https://cmssdt.cern.ch/jenkins/job/cms-bot/2181354/parameters/ and think that caused the re-processing of files and comments. I will check why 2nd job was started with v1

[smuzaffar]

yes if new bot v2 process ( e.g. https://cmssdt.cern.ch/jenkins/job/cms-bot/2181356/console ) the same PR then it does not re-read files and also does not edit the comment.

Problem is that for some of the github web hooks we do not know the PR details ( e.g. when we update commit status) and we end up using v1

[smuzaffar]

looks good, properly processed PR cms-data/MagneticField-Interpolation#8 . For more testing , lets get this merged and use it for already opened PRs with v2

[smuzaffar]

@iarspider , do you have any pending changes or can I go ahead and merge it?

[smuzaffar]

@iarspider , looks like new bot does not recognize enable = rntuple see #2893 where bot reported error ERRORS: key:workflows_rntuple; value:enable ( by the way error should be key: enable and value should be workflows_rntuple)

[iarspider]

@iarspider , looks like new bot does not recognize enable = rntuple see #2893 where bot reported error ERRORS: key:workflows_rntuple; value:enable

I will push a fix for that in a moment.

( by the way error should be key: enable and value should be workflows_rntuple)

I think the error message is the same in v1, no?

[cmsbuild]

Pull request #2891 was updated.

[smuzaffar]

I think the error message is the same in v1, no?

may be yes, but this should be fixed. error is for test parameters and they key there is enable and value is the test name

[iarspider]

@smuzaffar sure. Feel free to merge this once the checks pass.

[cmsbuild]

Pull request #2891 was updated.