Make dataset classes drop-in compatible with lerobot's dataset factory - #4
Merged
Conversation
`lerobot-train` instantiates its dataset via `make_dataset`, which calls `LeRobotDataset(repo_id, root=..., episodes=..., video_backend=..., ...)` and reads `dataset.absolute_to_relative_idx` when wiring the EpisodeAwareSampler. Three changes make both Lance classes usable there directly — `factory.LeRobotDataset = LeRobotLanceVideoDataset` is now the entire integration: - constructor signature matches upstream arg order: `repo_id` is the first positional, `root` second (a positional local path is still detected and treated as root for pre-0.2 call-style compatibility) - `absolute_to_relative_idx = None` class attribute: lance serves every frame by absolute index, so the upstream property (which would build a parquet reader) never applies - parquet-only factory kwargs (`video_backend`, `download_videos`, `force_cache_sync`, `depth_output_unit`) are accepted and ignored Verified with a 4-GPU accelerate run of lerobot-train (smolvla, episodes filter active) and the existing test suite (the one failing test fails identically on main under lerobot 0.6.0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AyushExel
added a commit
to lancedb/training
that referenced
this pull request
Jul 10, 2026
…-lancedb#4) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…and bump to 0.2.0 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…args in test fixtures Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
lerobot-trainhas no hook to swap its dataset class, so integrating the plugin previously required a user-side subclass shadowingabsolute_to_relative_idxplus an adapter function for the constructor signature. This PR moves that interface responsibility into the plugin, where it belongs. After it, the entire training integration is:Changes (both
LeRobotLanceDatasetandLeRobotLanceVideoDataset)repo_idfirst positional,rootsecond, matchingLeRobotDataset(repo_id, root=...)as the factory calls it. A positional local path is still detected (Path(arg).is_dir()) and treated asroot, so pre-existing positional call sites keep working.absolute_to_relative_idx = Noneclass attribute — lance serves every frame by absolute dataset index; the upstream property exists only for episode-filtered parquet readers and would try to build one. Semantically correct for lance, and letsEpisodeAwareSamplerwiring read it safely.video_backend,download_videos,force_cache_sync,depth_output_unitaccepted and ignored (documented as no-ops for Lance reads).Verification
accelerate launchrun oflerobot-train(smolvla on the LIBERO lance dataset,--dataset.episodesfilter active): trains normally,data_s=0.002.(repo_id, root=...), keywordroot=, positional path, and factory-style with parquet-only kwargs.test_subtask_index_round_tripserrors identically onmainunder lerobot 0.6.0 (pre-existingmeta.subtasksAPI drift, unrelated).Context: this fell out of the end-to-end LIBERO example/blog (lancedb/training#6), where the user-side shim was the one wart in the migration story.
🤖 Generated with Claude Code