Screen reader compatibility

[chrstnb]

We're using Ink for gemini-cli and are seeing issues with compatibility with screen readers (e.g., google-gemini/gemini-cli#1553).

To address this, I started attempting to make changes to our repo, but then I figured it might actually make sense to start by adding support to ink. I'm working on getting feedback from screen reader users, but wanted to open up here first!

My questions are:

1. Would you all be open to me adding this support, especially in a way that won't affect the experience for non-screen reader-users?
2. Do you have any preference between adding this into the root rendering logic?

If no concerns about the above, I'll put up a PR and start getting feedback from screen reader users on the approach!

Initial proposal would look something like this:

Screen Reader Support

To enable screen reader support, you can either pass the isScreenReaderEnabled option to the render function or set the INK_SCREEN_READER environment variable to true.

render(<MyApp />, {isScreenReaderEnabled: true});

When screen reader support is enabled, Ink renders a text-based, screen-reader-friendly version of the UI instead of the visual one.

For example, this UI:

<Box borderStyle="single" flexDirection="column">
	<Box>
		<Text>Username</Text>
		<Box flexGrow={1} />
		<Text>guest</Text>
	</Box>
	<Box>
		<Text>Password</Text>
		<Box flexGrow={1} />
		<Text>guestPassword</Text>
	</Box>
</Box>

For a screen reader, the output will be:

Username
guest
Password
guestPassword

Accessibility Props

To create a more accessible UI, you can use the following props, which are only used when screen reader support is enabled.

accessibilityLabel

You can use the accessibilityLabel prop on <Box>, <Text>, and <Transform> components to provide a custom, screen-reader-specific output. This label will replace the component's children in the screen reader output.

<Box accessibilityLabel="A box with a custom label for screen readers">
	<Text>This text will be replaced by the label above.</Text>
</Box>

accessibilityRole

Use the accessibilityRole prop on a <Box> to define its role, such as a button or checkbox. This helps screen reader users understand the element's function.

Supported roles: 'button', 'checkbox', 'radio', 'radiogroup', 'list', 'listitem', 'menu', 'menuitem', 'progressbar', 'tab', 'tablist', table, 'timer', 'toolbar'.

<Box accessibilityRole="button">
	<Text>Save</Text>
</Box>

In screen reader mode will be:

button: Save

accessibilityState

Use the accessibilityState prop on a <Box> to describe its current state. This is useful for interactive components.

Supported states: checked, disabled, expanded, selected.

<Box accessibilityRole="checkbox" accessibilityState={{checked: true}}>
	<Text>Accept terms and conditions</Text>
</Box>

This will output as checkbox: Accept terms and conditions (checked) in screen reader mode.

[sindresorhus]

This looks good to me and would be a welcome contribution.

[sindresorhus]

Minor:

• I would include a hook to be able to access the isScreenReaderEnabled state in components when you need to render things differently.
• Needs a accessibilityHidden attribute to hide something from being included in accessibility.

• This will output as checkbox: Accept terms and conditions (checked) in screen reader mode.

  • I think the state should come before the description.
• Future: Ink could use some more built-in semantic components, which could help avoid a lot of accessibility property usage.

[sindresorhus]

Question: Should we use ARIA naming? The benefit is that it's standardized, but it's also not that well known. The downside is that if we use the naming, we are stuck with its API and naming, even when we think it's bad.

[chrstnb]

Awesome, these are very helpful callouts. I'm going to put up a PR tomorrow with some initial bones for this, perhaps starting off just hiding any ascii art and then can add ARIA naming. I'm definitely in favor of that approach btw!

[andyfeller]

@chrstnb : simply put, thank you for introducing this support and providing accessible experiences! 🦸‍♀️