Add CocoIndex + zvec video hybrid search example#2271
Open
Haleshot wants to merge 4 commits into
Open
Conversation
Adds a CocoIndex example that indexes local video files into zvec by scene. Each scene stores a CLIP keyframe vector, an FTS transcript field, and scalar metadata for path and time range. Querying stays direct through zvec and combines dense, FTS, and scalar filters. Signed-off-by: Srihari Thyagarajan <hari.leo03@gmail.com>
Adds a download script for the Blender open movie sample clips, attribution for the sources, and an offline generated fallback for pipeline checks. Video binaries stay out of the repo. Signed-off-by: Srihari Thyagarajan <hari.leo03@gmail.com>
Documents setup, sample video download, indexing, hybrid query examples, incremental behavior, and troubleshooting for the video_hybrid_search example. Signed-off-by: Srihari Thyagarajan <hari.leo03@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a first pass at a local-video CocoIndex + zvec example.
The example watches a local videos/ folder, cuts each video into short scenes, embeds one keyframe with CLIP, transcribes the audio with faster-whisper, and writes one row per scene into
zvec. Each row has a dense keyframe vector, a full-text transcript field, and scalar fields for the video path and time window. The query script stays on the zvec side and combines dense search, transcript FTS, and scalar filters (based on what the connector supports now).The CocoIndex part is the indexing and freshness layer: add or delete a clip and the scene rows follow; change the CLIP model and the embedding step reruns while the transcript step stays cached. If the vector size changes, zvec rebuilds the collection, which the README calls out.
The example is tested on two Blender open movie clips, downloaded at runtime rather than checked in, with an attribution file for the sources. There's also a generated offline fallback for people who just want to test the pipeline mechanics.
Not added to the docs examples gallery yet. I kept this PR focused on getting an example up and running; happy to add the docs/gallery wiring in a follow-up if this works!
Related: cocoindex-io/blog-proposal#1