Rethinking Bevy's AccessKit integration

[viridia]

I have long observed that in application development, accessibility is most frequently treated as an afterthought - something that the dev team will get around to "eventually". Worse, most developers - even professional UI developers - are unfamiliar with accessibility technologies. Many don't even know how to enable screen reading on their preferred development platform.

Some history: Back in 2014, I created a JavaScript widget library while working in the JS Systems Group at Google, working in cooperation with members of Google's accessibility team. The goal was to create widgets that had accessibility built into it's DNA, so that developers would get the proper behaviors "for free". This toolkit was eventually incorporated into Google Plus, Google Sheets, Google Accounts and others.

My current project is to do something similar for the Bevy game engine: to create a library of "headless" (that is, unstyled) widgets with the proper accessible behaviors. This is inspired by various libraries in the JavaScript world such as ReaKit and react-headless.

The goal here is twofold: first, to make accessibility the default behavior for developers using these widgets; and secondly, to provide developers an easy introduction to the process of running and testing their UIs with screen readers and assistive technologies.

However, I and others have realized that the current Bevy integration with AccessKit is not ideal for this task, particularly in light of the direction that Bevy has been evolving over the last several releases.

Bevy is developing a new templating system for defining UI layouts. This is based around the ECS (Entity Component System) design that is foundational to Bevy. In particular, templates are meant to be "composable" in the sense that a template for an entity can be customized or overridden by merging with another template. For example, a template for a "headless" dialog that provides dialog-like behaviors can be combined with one that provides styling, allowing a separation of concerns. Doing this requires that the template entities be designed in a way that they are composed of components and properties that can be overridden. For most Rust data types, this is trivial.

However, the current AccessKit integration wraps the AccessKit Node in a single ECS mega-component, whose properties are not public. Merging together properties from different sources requires bespoke logic, and isn't particularly well-defined.

The current thinking is to instead define individual ECS components that represent the various AccessKit properties, and then combine them into an AccessKit Node in the background.

Note that in posting this, I'm not asking AccessKit devs to do anything; I just wanted to let you know what we're thinking. I guess if I had any "ask", it would be better documentation for developers on how to test AccessKit in their own apps, so that we could link this in the Bevy docs. (Merely googling for "accessibility testing for native app developers" yields a torrent of product ads.)

[mwcampbell]

I think it would be better to do this discussion under the Bevy project, not AccessKit itself. Also pinging @ndarilek who did most of the work on the Bevy AccessKit integration.

[ndarilek]

Agreed. FWIW, the current integration was done more as a quick PoC. I've wanted to do another pass through and integrate with BSN, relations (whenever those land) and maybe even triggers.

I'd actually been considering starting a Bevy a11y working group but thought it might be mostly me and lots of interested observers, and I don't have much capacity for shouldering new efforts entirely on my own right now. Seems like I might have been wrong about that, though. If we make that happen, that'd probably be the place to continue this chat.