Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[GLUTEN-8393][VL] Fix the smj result mismatch issue #8394

Merged
merged 6 commits into from
Jan 3, 2025
Merged

[GLUTEN-8393][VL] Fix the smj result mismatch issue #8394

merged 6 commits into from
Jan 3, 2025
+93 −20

Conversation

JkSelf
Copy link
Contributor

@JkSelf JkSelf commented Jan 2, 2025 •
edited
Loading

What changes were proposed in this pull request?

  1. Enable full outer join in smj.
  2. Fix the inner join result mismatch caused by gluten's incorrect formulation of the join condition. Specifically, gluten incorrectly reorders the condition, such as changing "A = B and C = D" to "C = D and A = B", which leads to a mismatched final join result.
  3. Fix the TPC-DS and TPC-H result mismatch issue in anti, semi and full outer join in fix: Fix the MergeSource data lost issue facebookincubator/velox#11772 and fix: Fix smj result mismatch issue in semi, anit and full outer join facebookincubator/velox#11771
  4. Set the ci job to cover the smj unit test.

How was this patch tested?

Existing unit tests and locally run TPC-H and TPC-DS.

@JkSelf JkSelf requested a review from rui-mo January 2, 2025 03:45
@github-actions github-actions bot added CORE works for Gluten Core VELOX labels Jan 2, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 2, 2025

#8393

@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 2, 2025

Run Gluten Clickhouse CI on x86

rui-mo
rui-mo approved these changes Jan 2, 2025
View reviewed changes
Copy link
Contributor

@rui-mo rui-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Looks great!

@github-actions github-actions bot added the BUILD label Jan 2, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 2, 2025

Run Gluten Clickhouse CI on x86

@github-actions github-actions bot added the INFRA label Jan 2, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 2, 2025

Run Gluten Clickhouse CI on x86

@JkSelf JkSelf changed the title [GLUTEN-8393][VL] Enable full outer join and fix the result mismatch in inner join [GLUTEN-8393][VL] Fix the smj result mismatch issue Jan 2, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 2, 2025

Run Gluten Clickhouse CI on x86

@JkSelf JkSelf force-pushed the enable-smj-full-outer branch from 7fe4fd4 to f6fafe3 Compare January 3, 2025 03:36
@github-actions github-actions bot removed the BUILD label Jan 3, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jan 3, 2025

Run Gluten Clickhouse CI on x86

@JkSelf JkSelf merged commit f2fecae into apache:main Jan 3, 2025
48 of 49 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rui-mo rui-mo rui-mo approved these changes

Assignees
No one assigned
Labels
CORE works for Gluten Core INFRA VELOX
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@JkSelf @rui-mo