Conversation
github-actions
bot
added
size/M
docs-not-required
Summary of ChangesHello @discord9, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request establishes a robust compatibility testing framework within the Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a comprehensive compatibility testing framework, which is a great addition for ensuring stability across different versions of GreptimeDB. The implementation closely follows the design outlined in the RFC, with a clear three-phase approach (feature, verify, cleanup) and new sqlness interceptors (SINCE, TILL, IGNORE_RESULT) to support version-aware testing.
The code is well-structured, with new responsibilities cleanly separated into new modules like cmd/compat.rs, compatibility_runner.rs, version.rs, and the interceptors directory.
My review focuses on improving error handling and correctness in a few areas:
- Refactoring the
compatcommand'srunfunction to use idiomatic Rust error handling withResultand the?operator. - Improving the version comparison logic to more robustly handle potentially unparsable versions.
- Enhancing user-facing error messages in the new interceptors.
These changes should improve the robustness and maintainability of the new testing framework. Overall, this is a solid piece of work.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aefc28f23a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c5990ad874
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
discord9
force-pushed
the
test/sqlness_compat
branch
from
ee346ee to
51e27ed
Compare
discord9
force-pushed
the
test/sqlness_compat
branch
from
5d9eb55 to
c4b3682
Compare
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
discord9
force-pushed
the
test/sqlness_compat
branch
from
c3c9d96 to
896344b
Compare
Signed-off-by: discord9 <discord9@163.com>
Signed-off-by: discord9 <discord9@163.com>
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d8dda8d824
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ComparisonOp::Eq => lhs == rhs, | ||
| ComparisonOp::Ne => lhs != rhs, |
There was a problem hiding this comment.
ComparisonOp::Eq/Ne currently use raw enum equality (lhs == rhs), which treats Version::Current as different from an equivalent semantic version. In compat runs with --to=current, expressions like -- SQLNESS VERSION version == 1.0.0-beta.1 will be evaluated as false even when the current binary is exactly that version, while </> use semantic ordering; this inconsistency causes valid statements to be skipped (or invalid ones to run) whenever ==/!= is used with current.
Useful? React with 👍 / 👎.
| fn should_setup_etcd(self) -> bool { | ||
| matches!(self, Self::Feature) |
There was a problem hiding this comment.
Compat mode only enables setup_etcd for 1.feature, so distributed tests in 2.verify/3.cleanup depend on feature phase having already started etcd. If users filter to a verify-only distributed case (including the README’s 2.verify/... style filter), feature may run no distributed cases and never start etcd, yet verify still uses etcd-backed metasrv config (store_addrs is fixed), causing the phase to fail unless an external etcd is already running.
Useful? React with 👍 / 👎.
| if !self.case_dir.exists() { | ||
| return Ok(()); |
There was a problem hiding this comment.
run_phase returns success when case_dir does not exist, so a mistyped --case-dir (or running from an unexpected layout) can report “Compatibility tests passed” while executing zero cases. This is a false-positive test result and differs from bare/kube, which validate case directories up front.
Useful? React with 👍 / 👎.
| @@ -0,0 +1 @@ | |||
| ../distributed/common No newline at end of file | |||
| @@ -0,0 +1 @@ | |||
| ../distributed/common No newline at end of file | |||
| @@ -0,0 +1 @@ | |||
| ../distributed/common No newline at end of file | |||
| if !self.preserve_state { | ||
| println!("Stopping etcd"); | ||
| crate::util::stop_rm_etcd(); |
There was a problem hiding this comment.
CompatCommand always calls stop_rm_etcd() when --preserve-state is false, even if the selected cases never started a distributed environment or the run failed before etcd came up. That means a standalone-only compat run can still fail in cleanup on machines without Docker, and an earlier real test failure can be masked by this cleanup path. Cleanup should be conditional on whether this invocation actually launched etcd.
| pub fn to_semantic(&self) -> Option<SemverVersion> { | ||
| match self { | ||
| Version::Semantic(v) => Some(v.clone()), | ||
| Version::Current => Self::current_version_from_cargo(), |
There was a problem hiding this comment.
VERSION current is not the version of the binary under test" body="The VERSION interceptor resolves current from env!("CARGO_PKG_VERSION"), which is just the workspace version baked into sqlness-runner at compile time.
2 participants
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
What's changed and what's your intention?
run
cargo sqlness compat --from=1.0.0-beta.1 --to=current --preserve-stateto run compat test!perhaps also need ci to test previous few versions automatically
Still need more feature cases, but the general framework is this, the case dir might be a little to verbose, but that's the easiest way to impl this, and the general idea of compat test from rfc, and also
TEMPLATEsqlness interceptor is also neededdue to metasrv need to restart, must use etcd to run which in turn need docker to run test
PR Checklist
Please convert it to a draft if some of the following conditions are not met.