AccessPulse iOS

AccessPulse iOS is a prototype universal accessibility intelligence layer for mobile workflows.

It explores a simple but important idea:

Accessibility should not stop at reading labels. It should help users understand intent, recover from friction, and complete tasks with confidence.

The project combines:

• real-world perception
• UI understanding
• adaptive voice and haptic guidance
• continuity memory across steps and screens
• mission-level task tracking
• trust-aware fallback recommendations

Why this project exists

Today, accessibility support is often split across separate systems:

• camera tools describe the physical world
• screen readers read accessible UI metadata
• human-assistance tools fill the hardest gaps

The missing layer is intelligent guidance for difficult app experiences, especially when accessibility structure is incomplete or the task itself is confusing.

AccessPulse is a prototype of that missing layer.

What is in this repo

• Examples/AccessPulseDemo Reference demo experience for a banking-style transfer flow.
• Packages/AccessPulseAccessibilityKit Shared reasoning engine for summaries, UI inference, planning, continuity, and trust.

Current capabilities

• Live form-state understanding for a high-friction transfer flow
• UI understanding that infers likely structure, primary action, zones, and friction points
• Screenshot-based analysis using OCR and layout cues
• Camera-based surroundings scanning with OCR, nearby-person detection, and scene cues
• Adaptive guidance modes for different confidence and pacing needs
• Continuity memory so guidance can resume after interruption
• Mission planning across entry, review, and confirmation states
• Trust evaluation with explicit fallback recommendations

Architecture at a glance

1. Perception Read nearby text, people, and scene cues from the environment.
2. UI Understanding Infer structure, reading order, zones, and likely actions from app state and visible screen artifacts.
3. Guidance Turn that understanding into summaries, routes, checkpoints, and recovery steps.
4. Continuity Preserve where the user was, what changed, and how to resume safely.
5. Trust Decide how confident the system is and what fallback is safest.

See docs/Architecture.md for the fuller system breakdown.

Honest scope

This repo demonstrates a strong accessibility prototype and reference architecture.

It does not yet claim to:

• understand every third-party app
• replace VoiceOver
• solve all mobile accessibility problems

What it does show is the product shape and reasoning stack for a more intelligent accessibility layer on iOS.

Quick start

Open the reference demo:

• Examples/AccessPulseDemo/README.md

Explore the shared package:

• Packages/AccessPulseAccessibilityKit/README.md

Use the presentation script:

• Examples/AccessPulseDemo/DemoScript.md

Near-term roadmap

• Broader visual UI analysis beyond the current controlled flow
• Better region-to-role inference for unlabeled controls
• Stronger cross-screen task transfer between more screen types
• A package-first SDK story for third-party app teams
• More demo scenarios beyond banking and transfer flows

Positioning

The strongest way to describe this project is:

AccessPulse is a prototype accessibility intelligence layer that combines perception, UI interpretation, task memory, and trust-aware guidance into one assistive system.