Technical Showcase: Bridging the gap between Ruffle and Touch Screens via Shadow DOM penetration

[coocoodaegap]

Hi everyone,

I've been exploring ways to enhance the accessibility of archived Flash-based web content on mobile devices using Ruffle. In the process, I encountered several technical hurdles regarding Ruffle’s input handling, and I wanted to share the architectural solutions I implemented.

I've encapsulated the logic into an open-source engine called OmniPad.

The Technical Challenges:

1. Encapsulation: Ruffle's Shadow DOM makes traditional elementFromPoint ineffective for virtual cursors. I implemented a recursive penetration logic using elementsFromPoint to resolve the deepest target without breaking the headless architecture.
2. Input Fidelity: Simulating "Mouse Hover" and "Double-tap Drag" on a touch device required a custom gesture state machine and high-fidelity event synthesis (Pointer + Mouse + Keyboard).
3. Architecture: I chose a Headless Core approach. All mathematical calculations (deadzones, vectors, signal debouncing) are pure TypeScript, decoupled from the Vue 3 adapter. This allows the UI to be highly customizable via CSS/Slots while the logic stays robust.

Demonstration:

Play on mobile browser:

omnipad1.mp4

Play with X-Box gamepad:

omnipad2.mp4

Current Status:
The MVP is complete and published. Due to limited personal bandwidth, I’m moving the project into Maintenance Mode. I’m sharing this here in hopes it might help other developers struggling with mobile-web-emulation.

Repo: https://github.com/omnipad-js/OmniPad
Demo: https://omnipad-demo.coocoodaegap.com/

I'd love to hear your thoughts on the event simulation approach!