Video Platform — System Design Review

Full corrected diagram: Miro board (link)

---

My Assumptions

The PRD has some intentional ambiguity. Here's how I filled the gaps:

• Profanity filter censors, not just detects. The PRD says "censor" and the audience is children, so the system extracts audio, runs speech-to-text, and mutes flagged segments before transcoding. If profanity density is too high, the video is rejected.
• Preview-first. To hit the ≤15s target, the 360p version is made playable as soon as it's done. Higher qualities keep processing in the background.
• Captions are static. If a .vtt file is provided, it's stored and served as-is. No processing.
• Single S3 bucket with structured paths: videos/{video_id}/{360|720|1080}p.mp4, etc.

---

What's Wrong with the Original Design

Upload

• File goes through the BFF. For files up to 1 GB with 500 concurrent uploads, this kills the server. It becomes a memory and bandwidth bottleneck.
• 201 response returns "thumbnail" — but processing hasn't started yet. No thumbnail exists.
• No retry or cancel mechanism. A failed upload means starting from zero.

Processing

• Everything is synchronous and sequential. No queue, no parallelism, no way to scale. Can't meet the ≤15s preview target.
• Missing: 1080p transcode, poster image, thumbnail sprite, metadata extraction, profanity filter, captions handling, status tracking. That's most of the PRD.

Playback

• "After video is completely downloaded, user can playback." This is the biggest issue. A 10-min 1080p video could be hundreds of MB. The user waits for a full download before seeing anything. Violates the ≤2.0s start time target.
• Query-param URLs (?res=360) are not cache-friendly. The PRD asks for immutable, versioned URLs.
• 600s CDN cache conflicts with the ≤60s deletion requirement.
• No manifest. The PRD explicitly asks for one.

General

• No database anywhere. No message queue.

---

Proposed Architecture

Upload

1. Client validates locally (file type, size).
2. Client requests a pre-signed URL from the BFF.
3. BFF generates a video_id, creates a DB record (status: pending_upload), returns the URL + ID.
4. Client uploads directly to S3 via multipart upload — chunk retry, progress events, cancel via AbortMultipartUpload. No file data touches the BFF.
5. Client calls POST /videos/{video_id}/upload-complete. Backend updates status and enqueues the first processing job.

Processing

FastAPI backend with Celery (Redis as broker).

Sequential (each step depends on the previous):

1. Virus scan — rejects infected/corrupted files.
2. Metadata extraction — gets duration, dimensions, codec via FFmpeg. Determines which transcodes to produce.
3. Profanity filter — speech-to-text → analysis → generates clean audio track. Must run before transcoding so all versions use the clean audio.

Parallel (fan-out after profanity filter):

• Transcode 360p / 720p / 1080p
• Poster image (single frame via FFmpeg)
• Thumbnail sprite (contact sheet + VTT for timestamp mapping)

When everything finishes, the callback generates a manifest.json, saves it to S3, and marks the video as ready.

All outputs use FFmpeg with -movflags +faststart so the MP4 moov atom is at the beginning, enabling immediate playback via range requests.

Why Celery over Lambda: Video transcoding is CPU-intensive and can run for several minutes. Lambda's 15-minute hard timeout and limited compute make it risky for the ≤10min P95 target. Celery workers on properly sized instances give full control over resources, and chain + chord model the pipeline naturally.

Status flow: pending_upload → scanning → extracting_metadata → filtering → processing → preview_ready → ready (or failed / rejected).

Playback

The playback path is fully static — no backend involved.

1. Client polls GET /videos/{video_id}/status via BFF until preview_ready or ready.
2. Client fetches manifest.json from CDN (→ S3 on cache miss).
3. Player renders poster, shows quality selector, defaults to 720p.
4. HTML5 <video> makes range requests to the CDN. Browser fetches only what it needs to start decoding — no full download. faststart makes this work.
5. Quality switch: save currentTime, swap src, seek back. PRD says reloading is acceptable.
6. Seek preview: sprite sheet + VTT, fully client-side.
7. Captions: <track> element pointing to the VTT on CDN.

Cache: Immutable URLs → long TTL (days/weeks). Invalidation only on delete.

Delete

1. Client calls DELETE /videos/{video_id} via BFF → FastAPI.
2. Backend does three things: marks deleted in DB, deletes manifest.json from S3, purges CDN cache for the manifest URL.
3. Without a valid manifest, the player can't resolve any URLs — playback fails immediately.

Only the manifest needs purging. Video files expire naturally from CDN cache via TTL. Actual S3 cleanup happens async via a background job.

What I'd Explore Next

• S3 Event Notifications to trigger processing automatically on upload, removing the need for the client upload-complete callback.
• Observability — job duration per pipeline stage, failure rates, queue depth, playback start time, rebuffer rate.
• SSE or webhooks for real-time status updates instead of polling.