Skip to content

Commit 2265f57

Browse files
committed
docs: add consolidated how it works guide
1 parent 555617a commit 2265f57

File tree

6 files changed

+313
-55
lines changed

6 files changed

+313
-55
lines changed

README.md

Lines changed: 24 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -8,64 +8,46 @@
88

99
AccessKit makes it easier to implement accessibility, for screen readers and other assistive technologies, in toolkits that render their own user interface elements. It provides a cross-platform, cross-language abstraction over accessibility APIs, so toolkit developers only have to implement accessibility once.
1010

11-
## How it works
11+
## Documentation
1212

13-
### Data schema
13+
* [How AccessKit works (architecture + Quick Start)](docs/how-it-works.md): canonical walkthrough of the schema, adapters, and a runnable winit example.
14+
* [Tips for application developers](README-APPLICATION-DEVELOPERS.md): practical screen reader testing guidance.
15+
* [Crate documentation on docs.rs](https://docs.rs/accesskit/): API reference for each published crate.
1416

15-
The heart of AccessKit is a data schema that defines all the data required to render an accessible UI for screen readers and other assistive technologies. The schema represents a tree structure, in which each node is either a single UI element or an element cluster such as a window or document. Each node has an integer ID, a role (e.g. button, label, or text input), and a variety of optional attributes. The schema also defines actions that can be requested by assistive technologies, such as moving the keyboard focus, invoking a button, or selecting text. The schema is based largely on Chromium's cross-platform accessibility abstraction.
17+
## Platform adapters
1618

17-
The canonical definition of the schema is in the Rust programming language. We chose Rust for its combination of efficiency and an expressive type system. Representations of the schema in other programming languages and schema definition languages (such as JSON Schema or Protocol Buffers) can be generated from the Rust code.
19+
AccessKit ships adapters for the major desktop platforms plus Android, all written primarily in Rust:
1820

19-
The schema is defined in the [accesskit crate](https://crates.io/crates/accesskit).
21+
* [Android adapter](https://crates.io/crates/accesskit_android): Implements the Java-based Android accessibility API.
22+
* [macOS adapter](https://crates.io/crates/accesskit_macos): Bridges to AppKit's `NSAccessibility`.
23+
* [Unix adapter](https://crates.io/crates/accesskit_unix): Talks to the AT-SPI D-Bus interfaces through [`zbus`](https://github.com/dbus2/zbus).
24+
* [Windows adapter](https://crates.io/crates/accesskit_windows): Implements UI Automation.
2025

21-
#### Serialization
26+
Planned adapters include iOS and a web/canvas bridge. Each adapter retains the full accessibility tree while toolkits keep only the state they need.
2227

23-
When the toolkit is written in Rust or another language that can efficiently call functions implemented in Rust, such as C or C++, the data defined in the schema can be passed back and forth with no serialization overhead. In other cases, serialization may be used to minimize the overhead of language interoperability. Because the schema supports [serde](https://serde.rs/), each language binding can choose its own serialization format.
24-
25-
### Platform adapters
26-
27-
These are the libraries that implement the platform-specific accessibility APIs.
28-
29-
The interaction between the provider (toolkit or application) and the platform adapter is also inspired by Chromium. Because Chromium has a multi-process architecture and does not allow synchronous IPC from the browser process to the sandboxed renderer processes, the browser process cannot pull accessibility information from the renderer on demand. Instead, the renderer process pushes data to the browser process. The renderer process initially pushes a complete accessibility tree, then it pushes incremental updates. The browser process only needs to send a request to the renderer process when an assistive technology requests one of the actions mentioned above. In AccessKit, the platform adapter is like the Chromium browser process, and the UI toolkit is like the Chromium renderer process, except that both components run in the same process and communicate through normal function calls rather than IPC.
30-
31-
One notable consequence of this design is that only the platform adapter needs to retain a complete accessibility tree in memory. That means that this design is suitable for immediate-mode GUI toolkits, as long as they can provide a stable ID for each UI element.
32-
33-
The platform adapters are written primarily in Rust. We've chosen Rust for its combination of reliability and efficiency, including safe concurrency, which is especially important in modern software. Some future adapters may need to be partially written in another language.
34-
35-
The current released platform adapters are all at rough feature parity. They don't yet support all types of UI elements or all of the properties in the schema, but they have enough functionality to make non-trivial applications accessible, including support for both single-line and multi-line text input controls. They don't yet support rich text or hypertext.
36-
37-
The following platform adapters are currently available:
38-
39-
* [Android adapter](https://crates.io/crates/accesskit_android): This adapter implements the Java-based Android accessibility API.
40-
* [macOS adapter](https://crates.io/crates/accesskit_macos): This adapter implements the NSAccessibility protocols in the AppKit framework.
41-
* [Unix adapter](https://crates.io/crates/accesskit_unix): This adapter implements the AT-SPI D-Bus interfaces, using [zbus](https://github.com/dbus2/zbus), a pure-Rust implementation of D-Bus.
42-
* [Windows adapter](https://crates.io/crates/accesskit_windows): This adapter implements UI Automation, the current Windows accessibility API.
43-
44-
#### Planned adapters
45-
46-
* iOS
47-
* web (for applications that render their own UI elements to a canvas)
28+
| Platform | Adapter crate | Support status | Notes |
29+
| --- | --- | --- | --- |
30+
| Windows 10+ | `accesskit_windows` | Stable | Exposes UI Automation. |
31+
| macOS 12+ | `accesskit_macos` | Stable | Known issue: ListBox item selection state not reported yet (`platforms/macos/README.md:5-7`). |
32+
| Linux / *BSD (AT-SPI) | `accesskit_unix` | Stable | Requires choosing an async runtime feature (`platforms/unix/README.md:3-10`). |
33+
| Android (GameActivity) | `accesskit_android` | Stable | `InjectingAdapter` currently depends on GameActivity (`platforms/winit/README.md:14-16`). |
34+
| iOS | _Planned_ || Adapter not yet implemented. |
35+
| Web/canvas | _Planned_ || Adapter not yet implemented. |
4836

4937
### Adapters for cross-platform windowing layers
5038

51-
#### winit
52-
53-
The [AccessKit winit adapter](https://crates.io/crates/accesskit_winit) is the recommended way to use AccessKit with [winit](https://crates.io/crates/winit), the popular Rust windowing library.
39+
* [AccessKit winit adapter](https://crates.io/crates/accesskit_winit): Recommended entry point for Rust applications built on [winit](https://crates.io/crates/winit). It wraps the platform-specific adapters and exposes a single API surface for delivering tree updates and handling action requests.
5440

5541
### Language bindings
5642

57-
UI toolkit developers who merely want to use AccessKit should not be required to use Rust directly. In addition to a direct Rust API, AccessKit provides the following language bindings, covering both the data schema and the platform adapters:
43+
UI toolkit developers who don't want to call Rust directly can use:
5844

59-
* [C bindings](https://github.com/AccessKit/accesskit-c): The C API is implemented directly in Rust, and the header file is generated using [cbindgen](https://crates.io/crates/cbindgen). We provide both a Windows-specific example using the Win32 API directly and a cross-platform example using SDL.
60-
* [Python bindings](https://pypi.org/project/accesskit/): These bindings are implemented in Rust using [PyO3](https://pyo3.rs/). We provide a cross-platform example using Pygame.
45+
* [C bindings](https://github.com/AccessKit/accesskit-c): C header generated with [`cbindgen`](https://crates.io/crates/cbindgen) plus Win32 and SDL samples.
46+
* [Python bindings](https://pypi.org/project/accesskit/): Implemented via [PyO3](https://pyo3.rs/) with a cross-platform Pygame sample.
6147

6248
### Consumer library
6349

64-
Some of the code required by the platform adapters is platform-independent. This code is in [the AccessKit Consumer Crate](https://crates.io/crates/accesskit_consumer). In addition to platform adapters, this library may also be useful for implementing embedded assistive technologies, such as a screen reader running directly inside an application, for devices that don't have platform support for accessibility at all, such as game consoles and appliances.
65-
66-
### Documentation
67-
68-
We realize that most developers who might use AccessKit are not experts in accessibility. So this project will need to include comprehensive documentation, including a conceptual overview for developers that are learning about accessibility for the first time.
50+
Shared logic that every adapter needs lives in [the AccessKit Consumer Crate](https://crates.io/crates/accesskit_consumer). It can also power embedded assistive technologies, such as a screen reader that runs directly inside a game engine or appliance that lacks platform-level accessibility APIs.
6951

7052
## Contributing
7153

docs/how-it-works.md

Lines changed: 161 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,161 @@
1+
# How AccessKit Works
2+
3+
AccessKit provides a cross-platform abstraction over native accessibility APIs, so a UI toolkit can describe its UI tree once and let AccessKit expose that tree to every supported operating system. This document is the canonical source for the “How it works” narrative and a practical quick start.
4+
5+
## Architecture at a glance
6+
7+
### Data schema
8+
9+
At the heart of AccessKit is a schema that expresses an accessibility tree. Each [`Node`](../common/src/lib.rs) has a stable [`NodeId`], a [`Role`] (for example `Button`, `Label`, or `TextInput`), optional attributes, and the set of actions it and its children support. Trees are delivered as [`TreeUpdate`] structs so the toolkit can stream incremental changes instead of rebuilding the entire tree.
10+
11+
### Tree updates and actions
12+
13+
The UI process (“provider”) pushes an initial tree to AccessKit and follows up with smaller updates whenever nodes change. Assistive technologies request interactions such as focus changes or button invocations through AccessKit actions; the toolkit responds by mutating its state and emitting another `TreeUpdate`. Only the platform adapter keeps a complete copy of the tree, which makes this model friendly to immediate-mode toolkits that can re-create nodes on demand as long as IDs stay stable.
14+
15+
### Platform adapters
16+
17+
Platform adapters translate the shared schema to the host accessibility API:
18+
19+
- `accesskit_android` (Android accessibility bridge)
20+
- `accesskit_macos` (AppKit `NSAccessibility`)
21+
- `accesskit_unix` (AT-SPI over D-Bus)
22+
- `accesskit_windows` (UI Automation)
23+
24+
Adapters are primarily written in Rust. The [winit adapter](../platforms/winit) wraps the appropriate per-platform adapter so Rust applications using the popular windowing library only need to depend on one crate. Planned adapters include iOS and a web/canvas bridge.
25+
26+
### Language bindings
27+
28+
In addition to the Rust API, AccessKit publishes C and Python bindings. Each binding exposes the same schema (generated from the canonical Rust definitions) and forwards platform events to the native adapters.
29+
30+
### Consumer library
31+
32+
Shared, platform-independent logic that adapters need lives in the [`accesskit_consumer`](../consumer) crate. Advanced use cases—such as embedded assistive technologies that run inside a game or appliance—can build on this crate directly.
33+
34+
## Quick Start (Rust + winit)
35+
36+
The snippet below mirrors the runnable sample in [`platforms/winit/examples/simple.rs`](../platforms/winit/examples/simple.rs) and shows the pieces you need in every application: describe the tree, initialize the adapter before the window becomes visible, forward window events, and react to AccessKit requests.
37+
38+
### 1. Add the dependencies
39+
40+
```toml
41+
[dependencies]
42+
accesskit = "0.21"
43+
accesskit_winit = "0.29"
44+
# Pick the winit backend features your app needs.
45+
winit = { version = "0.30", default-features = false, features = ["x11", "wayland"] }
46+
```
47+
48+
### 2. Describe your UI tree
49+
50+
```rust
51+
use accesskit::{Action, Node, NodeId, Role, Tree, TreeUpdate};
52+
53+
const WINDOW_ID: NodeId = NodeId(0);
54+
const BUTTON_ID: NodeId = NodeId(1);
55+
56+
fn build_initial_tree() -> TreeUpdate {
57+
let mut window = Node::new(Role::Window);
58+
window.set_children(vec![BUTTON_ID]);
59+
window.set_label("AccessKit demo");
60+
61+
let mut button = Node::new(Role::Button);
62+
button.set_label("Press me");
63+
button.add_action(Action::Click);
64+
65+
TreeUpdate {
66+
nodes: vec![(WINDOW_ID, window), (BUTTON_ID, button)],
67+
tree: Some(Tree::new(WINDOW_ID)),
68+
focus: BUTTON_ID,
69+
}
70+
}
71+
72+
fn button_pressed_update() -> TreeUpdate {
73+
let mut button = Node::new(Role::Button);
74+
button.set_label("Clicked!");
75+
button.add_action(Action::Click);
76+
77+
TreeUpdate {
78+
nodes: vec![(BUTTON_ID, button)],
79+
tree: None,
80+
focus: BUTTON_ID,
81+
}
82+
}
83+
```
84+
85+
### 3. Create the adapter before showing your window
86+
87+
```rust
88+
use accesskit_winit::{Adapter, Event as AccessKitEvent, WindowEvent as AccessKitWindowEvent};
89+
use winit::{
90+
application::ApplicationHandler,
91+
event::WindowEvent,
92+
event_loop::{ActiveEventLoop, EventLoop, EventLoopProxy},
93+
window::{Window, WindowId},
94+
};
95+
96+
struct DemoApp {
97+
window: Option<Window>,
98+
adapter: Option<Adapter>,
99+
proxy: EventLoopProxy<AccessKitEvent>,
100+
}
101+
102+
impl ApplicationHandler<AccessKitEvent> for DemoApp {
103+
fn resumed(&mut self, event_loop: &ActiveEventLoop) {
104+
let attrs = Window::default_attributes()
105+
.with_title("AccessKit demo")
106+
.with_visible(false);
107+
let window = event_loop.create_window(attrs).expect("window");
108+
let adapter = Adapter::with_event_loop_proxy(event_loop, &window, self.proxy.clone());
109+
window.set_visible(true);
110+
self.window = Some(window);
111+
self.adapter = Some(adapter);
112+
}
113+
114+
fn window_event(&mut self, _: &ActiveEventLoop, _: WindowId, event: WindowEvent) {
115+
if let (Some(window), Some(adapter)) = (&self.window, &mut self.adapter) {
116+
adapter.process_event(window, &event);
117+
}
118+
}
119+
120+
fn user_event(&mut self, _: &ActiveEventLoop, event: AccessKitEvent) {
121+
use accesskit::{Action, ActionRequest};
122+
123+
let adapter = match &mut self.adapter {
124+
Some(adapter) => adapter,
125+
None => return,
126+
};
127+
128+
match event.window_event {
129+
AccessKitWindowEvent::InitialTreeRequested => {
130+
adapter.update_if_active(build_initial_tree);
131+
}
132+
AccessKitWindowEvent::ActionRequested(ActionRequest { action, target, .. })
133+
if target == BUTTON_ID && action == Action::Click =>
134+
{
135+
adapter.update_if_active(button_pressed_update);
136+
}
137+
AccessKitWindowEvent::AccessibilityDeactivated => {}
138+
_ => {}
139+
}
140+
}
141+
}
142+
143+
fn main() -> winit::error::EventLoopError {
144+
let event_loop = EventLoop::with_user_event().build()?;
145+
let proxy = event_loop.create_proxy();
146+
let mut app = DemoApp {
147+
window: None,
148+
adapter: None,
149+
proxy,
150+
};
151+
event_loop.run_app(&mut app)
152+
}
153+
```
154+
155+
### 4. Keep tree updates incremental
156+
157+
- Call `adapter.process_event` for every winit `WindowEvent` so AccessKit can keep platform state in sync.
158+
- Whenever your UI changes, call `adapter.update_if_active` with a closure that returns only the nodes that changed plus the latest focus target.
159+
- Provide a full tree in the first update after `InitialTreeRequested`. Subsequent updates can set `tree: None`.
160+
161+
For more advanced scenarios (direct handlers, custom runtimes, or adapters outside winit), start from this document and explore the crate-level documentation on [docs.rs](https://docs.rs/accesskit/).

platforms/android/README.md

Lines changed: 35 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,40 @@
11
# AccessKit Android adapter
22

3-
This is the Android adapter for [AccessKit](https://accesskit.dev/).
3+
This crate exposes [AccessKit](https://accesskit.dev/) trees to Android’s accessibility framework.
44

5-
This adapter is implemented in two layers:
5+
## Adding the dependency
66

7-
* The `Adapter` struct is the core low-level adapter. It provides maximum flexibility in the application threading model, the interface between Java and native code, and the implementation of action callbacks, at the expense of requiring its caller to provide glue code.
8-
* The `InjectingAdapter` struct injects accessibility into an arbitrary Android view without requiring the view class to be modified, at the expense of depending on a specific Java class and providing less flexibility in the aspects listed above.
7+
```toml
8+
[target.'cfg(target_os = "android")'.dependencies]
9+
accesskit_android = { version = "0.4", features = ["embedded-dex"] }
10+
```
911

10-
The most convenient way to use `InjectingAdapter` is to embed a precompiled `.dex` file containing the associated Java class and its inner classes into the native code. This approach requires the `embedded-dex` Cargo feature.
12+
Enable the optional `embedded-dex` feature to bundle the helper Java class (`dev.accesskit.android.Delegate`) directly into your native library. If you disable the feature you must package that class yourself (for example via Gradle).
13+
14+
## Two integration layers
15+
16+
- [`Adapter`](src/adapter.rs) is the low-level bridge. You embed it inside your own `View` subclass and call its methods from JNI when Android invokes `AccessibilityNodeProvider`.
17+
- [`InjectingAdapter`](src/inject.rs) installs an `AccessibilityDelegate` and hover listener onto an existing `View`, posts callbacks to the UI thread, and forwards the framework requests to the low-level adapter. This is the simplest path when you can obtain a `View` handle (for example GameActivity’s surface).
18+
19+
## Using the low-level `Adapter`
20+
21+
1. **Store the adapter and handlers in your view.** Create `Adapter` plus `ActivationHandler` and `ActionHandler` instances when the host view is constructed.
22+
2. **Forward framework queries.** Implement `createAccessibilityNodeInfo`, `findFocus`, `performAction`, and hover handling in Java/Kotlin, then call the corresponding Rust methods (`create_accessibility_node_info`, `find_focus`, `perform_action`, `on_hover_event`). Each method requires both the current `JNIEnv` and the host `View`.
23+
3. **Deliver tree updates.** Call [`update_if_active`](src/adapter.rs#L224) from any thread with a closure that builds a `TreeUpdate`. When it returns [`QueuedEvents`](src/event.rs#L305), attach to the UI thread and call `raise(env, host)` to emit the queued `AccessibilityEvent`s. Never hold locks that your `View` might need while raising events; TalkBack can synchronously call back into your delegate.
24+
4. **Handle action callbacks.** Your `ActionHandler` receives `ActionRequest` structs whenever TalkBack invokes a control. Apply the action to your UI, then call `update_if_active` again so the tree stays in sync.
25+
26+
## Using `InjectingAdapter`
27+
28+
```rust
29+
let mut adapter = InjectingAdapter::new(&mut env, &host_view, activation_handler, action_handler);
30+
adapter.update_if_active(|| build_tree_update());
31+
```
32+
33+
- The helper registers an `AccessibilityDelegate` and `OnHoverListener`. If the view already has one, constructing `InjectingAdapter` will panic.
34+
- All callbacks are posted to the UI thread via `View::post`, so you can create or drop the adapter from any thread.
35+
- When the adapter is dropped it removes the delegate automatically.
36+
37+
## Testing
38+
39+
- Enable TalkBack from Android settings or by holding both volume keys, then explore your UI with touch to ensure nodes, labels, and actions surface correctly.
40+
- Use `adb logcat` to capture `accesskit` logs while interacting with TalkBack if you need to confirm event flow.

0 commit comments

Comments
 (0)