feat(cli): implement import-v2 data import pipeline#7898
feat(cli): implement import-v2 data import pipeline#7898fengjiachun wants to merge 3 commits intoGreptimeTeam:mainfrom
Conversation
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
github-actions
bot
added
size/L
docs-not-required
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements data import functionality for the import-v2 command, enabling the restoration of data from snapshots. It introduces the CopySource struct and execute_copy_database_from function to facilitate the COPY DATABASE SQL execution. The update also includes extensive end-to-end tests for data roundtrips, incomplete snapshots, and schema filtering. Feedback focuses on optimizing performance by replacing redundant storage listings with manifest checks, as well as improving code readability and formatting.
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 646d7c1b4d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: jeremyhi <fengjiachun@gmail.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6d72ed73f4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if chunk.status == ChunkStatus::Skipped { | ||
| continue; | ||
| } | ||
| for schema in schemas { | ||
| validate_chunk_schema_files(chunk, schema, &actual_prefixes)?; |
There was a problem hiding this comment.
validate_data_snapshot only checks each selected schema via validate_chunk_schema_files, and that helper returns Ok(false) whenever no matching prefix is found in chunk.files; as a result, a chunk marked Completed but carrying an empty files list is treated as “no data” and silently skipped, so import can report success while dropping that chunk’s data. Add an explicit invariant check that Completed chunks must contain at least one data file before continuing.
Useful? React with 👍 / 👎.
1 participant
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
As gemini said
PR Checklist
Please convert it to a draft if some of the following conditions are not met.