MV Wall Sync

MV Wall Sync is a local multi-screen MV playback experiment. It plays one video across multiple browser player windows by treating the video as a virtual canvas and assigning each connected player a rectangular viewport inside that canvas.

It is a small tool, not a polished video-wall product. The interesting part is the model: players can join and leave dynamically, and the controller can arrange any number of surface rectangles over the source video.

Features

• Virtual video canvas generated from the selected video metadata.
• Draggable and resizable surface rectangles, with aspect-ratio-preserving resize.
• Dynamic player registration through WebSocket.
• Player viewport reporting, so the controller can size a surface from the real browser window ratio.
• Backend-authoritative project state and playback state.
• Unified scheduled playback using server epoch time plus client clock-offset sampling.
• Player readiness tracking before unified playback.
• Online/offline player status.
• Single audio-player assignment; non-audio players stay muted to avoid doubled sound.
• Uploaded video mode and backend-local file mode.
• Backend video serving with byte-range support and ETag.
• OpenAPI schema generation for frontend types.

Architecture

The repository follows a Rust backend plus Vite frontend layout:

.
├── src/                  # Rust actix-web backend
│   ├── protocol.rs       # API/WebSocket DTOs and OpenAPI schemas
│   ├── state.rs          # Authoritative project and playback state
│   └── routes/           # API, video, OpenAPI, and frontend routes
├── frontend-panel/       # React/Vite/Tailwind controller and player UI
│   ├── src/core/         # Frontend project/layout/sync helpers
│   ├── src/services/     # API access and generated OpenAPI types
│   └── src/ui/           # Controller and player screens
└── tests/                # OpenAPI generation test

The backend is the authority. The controller and players keep local UI state, but playback, project updates, player registration, media readiness, and online/offline state flow through the Rust server.

BroadcastChannel exists only as a same-browser fallback when the backend WebSocket is unavailable.

Requirements

• Rust toolchain compatible with the Cargo.toml rust-version.
• Bun, used as the frontend package runner.
• ffprobe available in PATH when using local video path mode.

Development

Install frontend dependencies:

cd frontend-panel
bun install

Run the Rust backend:

cargo run

By default the backend listens on 127.0.0.1:8788.

You can override the backend host and port:

MV_WALL_SERVER_HOST=0.0.0.0 cargo run
MV_WALL_SERVER_PORT=8790 cargo run
MV_WALL_SERVER_HOST=0.0.0.0 MV_WALL_SERVER_PORT=8790 cargo run

Run the Vite frontend in another terminal:

cd frontend-panel
bun run dev

The Vite dev server listens on 127.0.0.1:8787 and proxies /api and WebSocket traffic to 127.0.0.1:8788.

Open:

• Controller: http://127.0.0.1:8787/
• Player: http://127.0.0.1:8787/?view=player

Production-Style Local Run

Build the frontend:

cd frontend-panel
bun run build

Then run the backend:

cargo run

The backend serves the built frontend from frontend-panel/dist:

• Frontend: http://127.0.0.1:8788/
• API state: http://127.0.0.1:8788/api/v1/state
• Video stream: http://127.0.0.1:8788/api/v1/video
• WebSocket: ws://127.0.0.1:8788/api/v1/ws

Basic Workflow

1. Open the controller.
2. Select a video.
3. Open one or more player windows.
4. Put each player on the physical display that should render that crop.
5. Assign players to surface rectangles in the controller.
6. Drag or resize the rectangles over the virtual video canvas.
7. Wait until all assigned players are online and media-ready.
8. Click unified play.

During playback, the controller progress bar shows the backend-authoritative playback position. Clicking or dragging the progress bar sends a seek request to the backend.

Video Sources

Upload Mode

The controller can upload a video to the backend. Uploaded video data is stored under:

data/video/

The backend serves the uploaded video at:

/api/v1/video

Local Path Mode

The controller can also tell the backend to use an absolute local file path. The backend probes metadata with ffprobe and serves the file directly through the same /api/v1/video endpoint.

This mode is useful when the controller, backend, and players are on the same machine, or when the backend machine has direct access to the media file.

Playback Model

Playback is backend-authoritative:

• The controller sends transport requests such as play, pause, restart, or seek.
• The backend updates WallProject.playback.
• Players follow the broadcast project playback state.
• Scheduled playback uses server epoch timestamps.
• Clients periodically sample server time and estimate serverClockOffsetMs.

When a new player joins during playback, it loads the current video, seeks to the backend-authoritative time, and attempts to play. If the browser blocks audible autoplay, the player shows an unlock button.

Audio Model

Only one player should output audio. The selected audio player is stored in project.audioPlayerId.

Non-audio players are muted so that multiple displays do not produce overlapping sound. This matters because even small playback drift between audible browsers is very obvious.

OpenAPI and Frontend Types

Rust protocol types are the source of truth. Frontend API/project types should come from generated OpenAPI types, not handwritten DTO copies.

Generate OpenAPI and frontend types:

cargo test --features openapi generate_openapi
cd frontend-panel
bun run generate-api

OpenAPI is served in debug builds with the openapi feature:

cargo run --features openapi

Then open:

http://127.0.0.1:8788/api-docs/openapi.json

Validation

Recommended checks:

cargo fmt --all --check
cargo clippy --all-targets --all-features
cargo test video::tests
cd frontend-panel
bun run check
bun run build

For a broader backend check:

cargo check
cargo check --features openapi

Notes and Current Limits

• The server is local-first and binds to 127.0.0.1 by default.
• The frontend dev server uses Vite proxying; production-style use is served by the Rust backend after bun run build.
• Browser autoplay policy still applies. Muted players can usually autoplay; the selected audio player may require one user click.
• Player synchronization depends on reasonable local clock-offset sampling and browser media behavior. The backend is authoritative, but the browser still controls actual decode/playback timing.
• ffprobe is required only for backend-local path mode.

LICENSE

MIT License