Skip to content

Conversation

wForget
Copy link
Member

@wForget wForget commented Sep 29, 2025

Which issue does this PR close?

Closes #2475.

Rationale for this change

Support full scalar/array arguments for rpad/read_side_padding

What changes are included in this PR?

Use make_scalar_function to convert arguments to arrays.

How are these changes tested?

unit test

@codecov-commenter
Copy link

codecov-commenter commented Sep 29, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.40%. Comparing base (f09f8af) to head (d57ea64).
⚠️ Report is 560 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2482      +/-   ##
============================================
+ Coverage     56.12%   58.40%   +2.28%     
- Complexity      976     1436     +460     
============================================
  Files           119      146      +27     
  Lines         11743    13510    +1767     
  Branches       2251     2350      +99     
============================================
+ Hits           6591     7891    +1300     
- Misses         4012     4389     +377     
- Partials       1140     1230      +90     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@wForget
Copy link
Member Author

wForget commented Sep 29, 2025

Failed spark test case:

[info] - SPARK-48498: always do char padding in predicates *** FAILED *** (330 milliseconds)
[info]   org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 605.0 failed 1 times, most recent failure: Lost task 0.0 in stage 605.0 (TID 582) (735d43d611a7 executor driver): org.apache.comet.CometNativeException: Unsupported data type (Dictionary(Int32, Utf8), Utf8) for function rpad/read_side_padding.
[info] This issue was likely caused by a bug in DataFusion's code. Please help us to resolve this by filing a bug report in our issue tracker: https://github.com/apache/datafusion/issues
[info] 	at org.apache.comet.Native.executePlan(Native Method)

@wForget wForget marked this pull request as draft September 29, 2025 12:28
@wForget wForget marked this pull request as ready for review September 30, 2025 11:35
@comphead
Copy link
Contributor

@coderfender as were working on rpad/lpad

@wForget
Copy link
Member Author

wForget commented Oct 15, 2025

@coderfender as were working on rpad/lpad

Thank you for the info, feel free to take over this.

@wForget wForget marked this pull request as draft October 15, 2025 14:10
@coderfender
Copy link
Contributor

@wForget , If my understanding is right, you are adding support to non scalar args to rpad function is it ? I can take a stab at the PR this weekend , fix the merge conflicts and make changes necessary to match spark's behavior and I hope that is Ok ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unsupported arguments [Array(StringArray[..]), Scalar(Int32(1)), Array(StringArray[..])] for function rpad/read_side_padding

4 participants