-
Notifications
You must be signed in to change notification settings - Fork 1.5k
[EPIC] A collection of tickets for improved WASM support in DataFusion #13815
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Dear @alamb , I hope you’re doing well. I’m very interested in contributing to the WASM support for DataFusion project as part of GSoC 2025. Enhancing embeddability and ensuring robust WASM integration aligns with my passion for building reliable and maintainable software. I’d greatly appreciate it if you could share more details on the project’s current challenges and key milestones. Understanding these aspects will help me prepare a thoughtful and well-structured proposal. Thank you for your time and guidance. Best regards, |
Thank you @savaliyabhargav I think the key challenge is the same as in this ticket's description:
I suggest some key milestones:
|
On this point, I was able to get WASM UDFs working in dft (im sure there is room for improvement but i got both WASM Native and Arrow IPC formats working) and as part of the next release I was planning to publish that crate so others could use it. @savaliyabhargav I would be happy to collaborate on that if you are interested in it. |
@matthewmturner yes sure i am interested can you please give me more detail about it |
For the WASM UDFs they just need some more real world testing / benchmarking. To be honest, the other points @alamb mentioned would probably better benefit the DataFusion ecosystem at this stage. But, if WASM UDFs are what interest you then you could make an issue on the dft repo and we could discuss some tangible next steps for that. |
Hi, I'm also interested in working on this project for Gsoc 2025. I'm currently a master student at Georgia Institute of Technology, I had some experiences with WASM and Rust in my undergrad study, and I would like to see how that can be incorporated with DataFusion! Are there any prerequisite for this project? I suppose the points below would be the key points for our proposal, right?
I'm new to DataFusion but I've heard of the name for a long long time.
Thanks in advance! |
@matthewmturner Please provide me with some information on DFT so that I can learn and work on your project effectively |
Is your feature request related to a problem or challenge?
DataFusion can be compiled today to WASM with some care. However it is somewhat tricky and the experience could be improved.
Describe the solution you'd like
This ticket tracks various ways we could make the experience better
Describe alternatives you've considered
datafusion-core
) to WASM #7652wasm
target #15599 (comment)Additional context
No response
The text was updated successfully, but these errors were encountered: