push_to_hub does not upload videos

[DominikVincent]

Describe the bug

Hello,

I would like to upload a video dataset (some .mp4 files and some segments within them), i.e. rows correspond to subsequences from videos. Videos might be referenced by several rows.

I created a dataset locally and it references the videos and the video readers can read them correctly. I use push_to_hub() to upload the dataset to the hub.

Expectation: A user uses load_dataset and can load the videos.

However, the videos seem to be just referenced via paths on the computer and not uploaded to the hub. Therefore a target user cannot load the videos in the dataset.

Steps to reproduce the bug

1. create a video dataset with paths e.g. { ["videos"]: [path1, path2, ...]}
2. dataset.push_to_hub
3. on a different computer (or same pc if relative paths are used in a different folder):

dataset = load_dataset("siplab/egosim", split="train")
video = dataset[0]["video_head"]

3. will fail

Expected behavior

Expectation: A user uses load_dataset and can load the videos.

Environment info

datasets 3.1.0
Python 3.8.18

[lhoestq]

Hi ! the Video type is still experimental, and in particular push_to_hub doesn't upload videos at the moment (only the paths).

There is an open question to either upload the videos inside the Parquet files, or rather have them as separate files (which is great to enable remote seeking/streaming)

[phineas-pta]

im having the same issue (btw i mistook this to be xet error https://huggingface.co/spaces/xet-team/README/discussions/4 )

@jsulz suggested me to use upload_folder but it exceeds hf limits (>10k files per folder and >100k files in total)

from my reading of the docs, in my case i have to save as either parquet or webdataset and then use upload_folder

i tried ds.to_parquet("...") but the parquet file also doesnt contain video, as of datasets v4.0

so i think the only workaround for my case is webdataset

[lhoestq]

in that case you can create a VideoFolder dataset instead, see the docs at http://huggingface.co/docs/datasets/video_dataset#videofolder

[lhoestq]

I just added support in push_to_hub() for videos :)

Note a small discrepancy for video datasets I hope to fix soon:

• load_dataset("username/my-folder-of-videos", streaming=True) -> videos are lazy loaded one by one when iterating, and only actually downloaded when accessing frames in torchcodec (only the requested frames are downloaded)
• load_dataset("username/my-video-dataset-in-parquet-saved-with-push_to_hub", streaming=True) -> videos are downloaded one by one when iterating. Even if no frame is accessed in torchcodec, the videos are downloaded completely