Skip to content

Run object_store tests (on CI) with WASM build/runtime #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Tracked by #6818
alamb opened this issue Jan 14, 2025 · 2 comments
Open
Tracked by #6818

Run object_store tests (on CI) with WASM build/runtime #26

alamb opened this issue Jan 14, 2025 · 2 comments
Labels
enhancement New feature or request

Comments

@alamb
Copy link
Contributor

alamb commented Jan 14, 2025

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

As we improve the story for using object_store with WASM we currently check that the code compiles with WASM, but have no coverage for actually running tests iina. WASM runtime to verify behavior. For example, as @XiangpengHao points out we have no way to verify support for files larger than 4GB (where WASM's 32-bit usize can be a problem):

Describe the solution you'd like
I would like to run all relevant tests from object_store on a wasm runtime to ensure they actually run

Describe alternatives you've considered
I don't really know much about how this works, but I can imagine potentially using this crate:

https://github.com/bytecodealliance/wasmtime

Somehow to run the generated wasm

Additional context

@alamb alamb added the enhancement New feature or request label Jan 14, 2025
@alamb
Copy link
Contributor Author

alamb commented Jan 14, 2025

@Xuanwo provided an example of how they do it in opendal: https://github.com/apache/opendal/blob/781682cc05b31d1f12f91fb7568a868f038c4d3b/.github/workflows/test_edge.yml#L62-L107

Thanks for the review @alamb , I agree with the rationale but unfortunately we don't yet have the infra to test wasm32 yet. We currently only build on wasm32, but there's no runner that can actually execute the webassembly

OpenDAL has an edge test for this; it's worth taking a look.

https://github.com/apache/opendal/blob/781682cc05b31d1f12f91fb7568a868f038c4d3b/.github/workflows/test_edge.yml#L62-L107

The test code could be found here: https://github.com/apache/opendal/tree/main/core/edge/s3_read_on_wasm

@alamb
Copy link
Contributor Author

alamb commented Mar 20, 2025

Migrating from arrow-rs issue #6980

@alamb alamb transferred this issue from apache/arrow-rs Mar 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant