Skip to content

[test][button] Add axe test coverage and docs#48708

Open
mj12albert wants to merge 3 commits into
mui:masterfrom
mj12albert:docs-button-axe-coverage
Open

[test][button] Add axe test coverage and docs#48708
mj12albert wants to merge 3 commits into
mui:masterfrom
mj12albert:docs-button-axe-coverage

Conversation

@mj12albert

@mj12albert mj12albert commented Jun 22, 2026

Copy link
Copy Markdown
Member
Result Count
✅ Supports 23
⚠️ Partially Supports 4
❌ Does Not Support 0
➖ Not Applicable 28
🚩 Unverified 17/27

Doc: https://github.com/mj12albert/material-ui/blob/docs-button-axe-coverage/packages/mui-material/src/Button/accessibility.md

Expand axe test coverage for Button to cover as much of WCAG as possible, and document the component's current WCAG conformance and testing notes/procedures.

@mj12albert mj12albert added test scope: button Changes related to the button. labels Jun 22, 2026
@code-infra-dashboard

code-infra-dashboard Bot commented Jun 22, 2026

Copy link
Copy Markdown

Deploy preview

https://deploy-preview-48708--material-ui.netlify.app/

Bundle size

Bundle Parsed size Gzip size
@mui/material 0B(0.00%) 0B(0.00%)
@mui/lab 0B(0.00%) 0B(0.00%)
@mui/private-theming 0B(0.00%) 0B(0.00%)
@mui/system 0B(0.00%) 0B(0.00%)
@mui/utils 0B(0.00%) 0B(0.00%)

Details of bundle changes


Check out the code infra dashboard for more information about this PR.

@mj12albert mj12albert force-pushed the docs-button-axe-coverage branch 6 times, most recently from 99db26a to 014886f Compare June 25, 2026 08:02
@mj12albert mj12albert marked this pull request as ready for review June 25, 2026 08:05
@siriwatknp

Copy link
Copy Markdown
Member

@mj12albert the accessibility.md is not auto-generated right? I want to get alignment on the plan to make them public on the docs, what's your view on this?

@mj12albert

mj12albert commented Jun 25, 2026

Copy link
Copy Markdown
Member Author

the accessibility.md is not auto-generated right?

No ~ it's impossible to auto-generate (at least now) 😅, I kick-started research with AI but eventually had to manually re-do ~80% of it anyway, so for each I gave it my manual notes and initial draft as a start

Possibly once we have a few more of these, there will be more reference material that improves the results, but the fundamental issue is that AI is unreliable at "understanding" the WCAG specs (esp when there's no direct code to trace)

Possibly we can later automate the aggregation of component level scorecards:

| Result                | Count |
| :-------------------- | :---- |
| ✅ Supports           | 23    |
| ⚠️ Partially Supports | 4     |
| ❌ Does Not Support   | 0     |
| ➖ Not Applicable     | 28    |

into a library-level scorecard

I want to get alignment on the plan to make them public on the docs

I agree we should do this eventually, but that's a matter of presentation; for now I'm solely focused on scoring components, first so there is a body of content to present. IMO we don't need to worry about this until we have ~20 components scored. Does that make sense? @siriwatknp

Comment on lines +75 to +77
## Reports

- [Button](./Button/accessibility.md)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of individual reports I would like a table, where the findings are aggregated into a per level (A / AA) report. This could effectively look similar to this:

Component WCAG 2.2 A WCAG 2.2 AA
Button ✅ (8/8 ) ⚠️ (5/12)

...

@mj12albert mj12albert Jun 25, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the component level ones can eventually be aggregated into being displayed something like this, though x/total AA may not accurately partial/irrelevant criteria, so far I'm just following a somewhat standard formats/terminology

Comment thread test/regressions/demoMeta.ts Outdated
export const A11Y_RULES: A11yRule[] = [
{ test: 'docs/data/material/components/buttons/{BasicButtons,ColorButtons}', enabled: true },
{
test: 'docs/data/material/components/buttons/{BasicButtons,TextButtons,ContainedButtons,DisableElevation,OutlinedButtons,ColorButtons,ButtonSizes,IconLabelButtons,InputFileUpload,LoadingButtons,CustomizedButtons,ButtonA11yNonNative,ButtonA11ySemanticStates,ButtonA11yTextSpacing}',

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pretty hard to read. Can we maybe use an array of strings here that we define outside of the scope and just spread it here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, I think quite soon we can't pack everything in a single file either or it'll get ugly

@mj12albert mj12albert force-pushed the docs-button-axe-coverage branch 4 times, most recently from 2edd27f to 7bd80c7 Compare June 26, 2026 00:36
@mj12albert mj12albert force-pushed the docs-button-axe-coverage branch from 7bd80c7 to 4654ad4 Compare June 26, 2026 03:33
@siriwatknp

Copy link
Copy Markdown
Member

focused on scoring components

fair enough.


#### 1.3.3 Sensory Characteristics · A

`🚩 Unverified` · `✅ Supports` · `○ Author`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Unverified confuses me or at least better to present differently. In my understanding it means "not audited by trusted firms" right?

If yes, I think it's better to be removed and declare globally in accessibility.md instead on how we do audit manually.

Currently there are too many "Unverified" which is expected because we don't plan to do external audit soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

scope: button Changes related to the button. test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants