-
Notifications
You must be signed in to change notification settings - Fork 204
fix: fall back on nested types for default values #1799
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
spark/src/main/scala/org/apache/comet/rules/CometScanRule.scala
Outdated
Show resolved
Hide resolved
Co-authored-by: Andy Grove <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @mbutrovich
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #1799 +/- ##
============================================
+ Coverage 56.12% 59.44% +3.31%
- Complexity 976 1143 +167
============================================
Files 119 128 +9
Lines 11743 12538 +795
Branches 2251 2357 +106
============================================
+ Hits 6591 7453 +862
+ Misses 4012 3892 -120
- Partials 1140 1193 +53 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
Which issue does this PR close?
Partially address #1545.
Rationale for this change
#1756 added default value support for native_datafusion scan. However, this only works for Literals. If the default values are nested types, we can't yet serialize those to the native side for the scan operator to use. This PR adds a fallback for that scenario.
What changes are included in this PR?
Enhanced CometScanRule to look for default values that are not Literals.
How are these changes tested?
Now passes Spark SQL tests: