Skip to content

[test] Fix flaky DocSearch modal screenshots - #49122

Merged
LukasTy merged 1 commit into
mui:masterfrom
LukasTy:claude/docsearch-argos-pointer
Sep 11, 2026
Merged

[test] Fix flaky DocSearch modal screenshots#49122
LukasTy merged 1 commit into
mui:masterfrom
LukasTy:claude/docsearch-argos-pointer

Conversation

@LukasTy

@LukasTy LukasTy commented Sep 11, 2026

Copy link
Copy Markdown
Member

The Argos captures of the DocSearch modal sometimes show the hover state of the MUI X "Overview" link on the start screen. The current master baseline of regression-AppSearch/SearchModalDarkOpen.png has it, and so does https://app.argos-ci.com/mui/material-ui/builds/51991/454471015.

The test now moves the pointer to the top-right corner right after the click that opens the modal. That point is outside the modal at both capture widths, so no hover state gets into a capture.

Extracted from #48983, so that it can merge before the DocSearch bump.

In CI, the Open capture sometimes shows the hover state of a link on the
start screen. Move the pointer to the top-right corner right after the
click that opens the modal. That point is outside the modal at both
capture widths, so no hover state gets into a capture.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasTy LukasTy added type: bug It doesn't behave as expected. test labels Sep 11, 2026
LukasTy added a commit that referenced this pull request Sep 11, 2026
The fix is now in #49122, so it
can merge before this bump. This restores the test file to its master
version.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@LukasTy LukasTy mentioned this pull request Sep 11, 2026
1 task
@code-infra-dashboard

Copy link
Copy Markdown

Deploy preview

https://deploy-preview-49122--material-ui.netlify.app/
QR code for https://deploy-preview-49122--material-ui.netlify.app/

Bundle size

Bundle Parsed size Gzip size
@mui/material 0B(0.00%) 0B(0.00%)
@mui/lab 0B(0.00%) 0B(0.00%)
@mui/private-theming 0B(0.00%) 0B(0.00%)
@mui/system 0B(0.00%) 0B(0.00%)
@mui/utils 0B(0.00%) 0B(0.00%)

Details of bundle changes


Check out the code infra dashboard for more information about this PR.

@LukasTy LukasTy self-assigned this Sep 11, 2026
@LukasTy

LukasTy commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Argos marked CrudDashboard.png as "added" on #49122 (https://app.argos-ci.com/mui/material-ui/builds/52015/456363749), but that PR does not touch it. The cause is how our nightly jobs upload to Argos.

What happens:

  • The nightly test_regressions-react@18 and test_regressions-react@next jobs upload with the same default build name as the normal test_regressions run.
  • Master auto-approves every build, and Argos uses the last approved build on the merge-base commit as the baseline. On 5ce5a0f, that was the nightly React 18 build: https://app.argos-ci.com/mui/material-ui/builds/52007
  • Since [test] Skip the CrudDashboard VRT on React 18 #49027, the React 18 run skips CrudDashboard. So that baseline does not have it, and PRs based on that commit show it as added.
  • Nightly runs can also change other baselines. Build 52007 changed SearchModalOpen.png too.

Proposal: give the non-stable React runs their own Argos build name. Argos only compares builds with the same name. @argos-ci/core reads ARGOS_BUILD_NAME from the environment, so code-infra and the orb need no change. In the test_regressions job, before code-infra/argos-push:

- when:
    condition:
      not:
        equal: [stable, << parameters.react-version >>]
    steps:
      - run:
          name: Use a separate Argos build name
          # Keeps the React canary runs out of the baseline that PRs compare against.
          command: echo 'export ARGOS_BUILD_NAME="$CIRCLE_JOB"' >> "$BASH_ENV"

Result: the nightly builds become test_regressions-react@18 and test_regressions-react@next, each with its own baseline and its own argos/<name> status. PR builds compare only with the normal master run. Argos also adds an argos/summary check on commits with more than one build.

Questions:

  1. Do we need the nightly screenshots in Argos at all? Master auto-approves them, so nobody has to review them. Skipping the upload for non-stable React runs is simpler and uses fewer screenshots, but we lose the nightly visual history.
  2. nightly-cron also runs on v5.x, v6.x, and v7.x. Do we want the same change there?

@LukasTy
LukasTy merged commit 62a348b into mui:master Sep 11, 2026
21 checks passed
@LukasTy
LukasTy deleted the claude/docsearch-argos-pointer branch September 11, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test type: bug It doesn't behave as expected.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants