-
Notifications
You must be signed in to change notification settings - Fork 208
Move wasm32 Custom RNG impls back to main crate #149
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
I am not closely familiar with Rust WASM ecosystem, so I can't comment on universality of such solution (e.g. should we care about users of older
Personally I believe we should keep it as an external RNG. Supporting
Sounds good to me. |
This was my thought as well, however it might be worth considering the alternative of having an (opt-in) "js" cargo feature. Both the feature and the external crate do the same thing, they say "my wasm32 target has JS". The feature would make things sightly easier for users. It's also similar to the "cpu" feature. Is there a reason to support one of these approaches over another? I agree that unconditionally assuming JS on wasm32 unknown is probably not the best way forward. EDIT: Having an opt-in |
I don't think there is a big difference between adding As for the |
I think this is a good point. Even if a CPU's RNG isn't available, we can always generate code to check for this availability. However, for |
The only (minor) downside to the second approach is that the user also has to write
EDIT: I think that whatever we do here will not affect what we do for |
I am not sure if I fully understood you here, but I think you forgot about the
Oh, you are right. I didn't know that
It was mostly about the "unconditional support" variant. One minor drawback of moving from the custom crate is that it would require whitelisting (unused) WASM crates, which is not ideal (i.e. as you said it would pollute |
This seems reasonable, but for how to interpret wasm32-unknown-unknown, I don't think there's really a great answer at this point. I think today it's almost exclusively used for browser use cases with wasm32-wasi dominating the out-of-browser use case, but it's unlikely that either of these remains true until the end of time. The ecosystem is still developing that I think there's basically not a great answer here. You could by-default assume JS with wasm32-unknown-unknown, but it's almost guaranteed to be an incorrect assumption eventually. You could also require an opt-in, but this will be unergonomic for practically all current users of wasm32-unknown-unknown (and into the future as well). |
I think it's better to have two features,
I think this is fundamentally wrong, and it also isn't very flexible: even with wasm-bindgen you might want different behavior on Node or the web. Or you might want to support other uses of Wasm, such as crypto or Amazon AWS. Targets just aren't flexible enough to handle the Wasm ecosystem, whereas Cargo features work well. Instead, I think the correct approach is to use Cargo features (e.g. Having a separate crate doesn't work as good, because it doesn't work with transitive dependencies. |
I don't think so. We can't define mutually exclusive features, so if one of your dependencies wrongly enables such feature (be it for convenience sake or as a mistake), it will infect all downstream crates. In other words, the more crates in your dependency tree the more likely situation that both
Can you elaborate? The idea behind the custom crates, is that they get included only in application crates or as a dev dependency for benches and tests. It's essentially similar to how |
It is possible to use And in any case, that also happens with multiple crates: what if one crate uses
If there is a library Even if my application installs |
It will result in a linking error with the following description
No, it will be affected. I think you misunderstand how custom crates work. With an enabled |
Ah, sorry, I had assumed that it was still using the 0.1 behavior. However, that approach won't work for all crates, and it's also not zero cost. Which is why other crates use Cargo features. So I think it's good to use the same system throughout the ecosystem, rather than creating ad-hoc systems.
That seems strictly worse than using
I don't really understand that argument: enabling a feature is an explicit opt-in change which people only do when they need to enable that feature. And it often requires changing the syntax from So how is slapping a dependency into your crate harder than enabling a feature? At worst they seem equally hard to me. |
@newpavlov @Pauan @alexcrichton thanks for the feedback here. I started playing around with things and I've noticed some usability issues with having Bad error messagesDue to how Custom RNGs are implemented (via This works great if you are only compiling for one target. However, Cargo features enabled on any target apply to all targets. This means that if you use a Custom RNG on any target, the Transitive Stability IssuesImagine we have a crate [dependencies]
getrandom = "0.2"
getrandom-js = { version = "0.2", optional = true}
[features]
wasm-bindgen = ["getrandom-js"] and #[cfg(feature = "wasm-bindgen")]
use getrandom_js as _; But now say a crate A similar compatibility issue occurs if a crate just wants to only support If ProposalUntil something like rust-lang/cargo#7914 fixes the per-target Cargo feature issue, using Custom RNGs will be possible, but awkward in practice (in that they will only work if defined in the root binary crate). I don't think that we should block releasing We should put the The final docs for fn my_rng(&mut buf) -> Result<(), getrandom::Error> {
// ...
}
register_custom_getrandom!(my_rng); This makes [dependencies]
getrandom = { version = "0.2", features = ["custom"]} Let me know your thoughts on the proposal, I will update this PR to reflect what I am proposing here. |
Some other comments on stuff in this thread:
@Pauan, while I agree that the JS stuff shouldn't be done via a custom crate, I'd like to know what sorts of use-cases are incompatible with the above
I don't think having two features here is necessary. The "types, tools, and APIs" are also very different between all the other targets |
Can you elaborate? As for zero-cost, firstly the cost is really negligible, especially when compared to the cost of calling into JS world. And secondly, if I am not mistaken LTO should be able to optimize such code.
Yes, it's less convenient and clear, but it could be worked around. (see below)
I think it's easier to wrongly enable feature for convenience sake (e.g. if you test you library crate on WASM targets) and since it will not affect most users it can easily go undetected. Another problem is that feature can be enabled anywhere in your dependency tree and it could be quite hard to find the source of the problem, while finding the crate which has added a custom crate is much easier. Also there is a problem of feature unification between dev and non-dev builds, granted it only affects crates in the same workspace (if I remember correctly), but still it could be quite annoying. Another problem is potential breaking changes in @josephlr First, about the Second, about the error issue. We could add special features to |
I fairly sure that regardless of what approach we take here, we can fix these issues with semver tricks (assuming not too much has changed).
I'm aware that crates usually won't directly depend on
Both of these approaches are already done by many crates in the ecosystem, and we should support them. I can't figure out how to support these two use cases with
I agree that we should try to mimic global allocators here. For example, We could take a similar approach. External crates would define a function (with signature |
I really dislike the fact that adding
If the registration macro will originate in |
Ya, I was initially concerned about this, but the fact that
I don't think we will be able to use the custom crate under the hood until the target-specific feature issue gets resolved (but it looks like there's a path to stabilization there). So I think we should have the impls in the main crate until that issue is fixed, then we can move to an external RNG crate.
That's a good point, I think users should be able to use a custom RNG crate without depending on |
Another point is that all of these deps only show up in [[package]]
name = "getrandom"
version = "0.2.0"
dependencies = [
"cfg-if",
"libc",
"wasi",
] which is the same as |
Good to know! I was afraid that it would be necessary to bring all those WASM dependencies in the vendoring scenario.
Hm, I am not sure how absence of target-specific features blocks moving WASM implementation into a separate crate. But I realized that my proposal unfortunately has a problem: I still don't quite like bringing WASM code back into |
They will be gated behind the "js" feature, as we can now do detect, at compile-time, which implementation (wasm-bindgen vs stdweb) we should use. The "js" implementation takes precedence over the "custom" implementation. This prevents issues that arise from the buggy way Cargo handles features across multiple targets. Signed-off-by: Joe Richey <[email protected]>
Add back the "test-in-browser" feature. This makes it easier to manage a single file containing all of the test code. Signed-off-by: Joe Richey <[email protected]>
I agree, this is my biggest concern moving forward. Helping users to update and watching large projects and seeing if this gets enabled by default will be interesting. Using a feature seems like a good short-term solution, and it might be a good long-term solution (or it might not). It mostly depends on if non-JS @newpavlov I've rebased this onto the latest |
It won't work for traits, it won't work if different types are needed for different targets, it won't work for methods, and it won't work for non-C types. The only reason it works for getrandom is because the type signature is so simple (
That's a good point, but the same can happen with custom crates as well (forgetting to put it into
How is it easier? You still have to search through all of the crates and look at their
I've never heard of that, could you explain more?
Why do you think that's a good thing? That sounds terrible to me! Having a crate being able to affect another crate like that is crazy spooky action at a distance. It's even worse that it affects the same crate but with different versions. That will lead to bizarre bugs which are incredibly hard to debug. The whole point of major version bumps is to create a clean break from the previous version. If a major version bump was done, the extern fn should be renamed so that it doesn't collide with previous versions. This adds a maintenance hazard which doesn't exist for Cargo features.
This sounds like it's adding even more ad-hoc complexity to workaround problems which don't exist with Cargo features. Why not just use Cargo features? They are literally designed for this use case, they are simple, they are easy, they don't have spooky action at a distance, they can be used for everything (not just C functions), they don't cause issues with versioning, and they have better error messages. The only benefit you have mentioned for separate crates is that they are potentially harder to accidentally enable (which I don't agree with, and you haven't provided much evidence for). |
Right now
getrandom 0.2
has two custom RNG crates:stdweb-getrandom
andwasm-bindgen-getrandom
. This is because the Rust community has two main ways to call JavaScript from Rust code onwasm32-unknown-unknown
(wasm-bindgen and cargo-web), so we support both. There's also the issue thatwasm32-unknown-unknown
targets aren't necessarily running in a JavaScript environment.However,
cargo web v0.6.24
added support for proper conditional compilation. This means we don't actually need two distinct crates, we can just "do the right thing" at compile time and only have a single crate.Furthermore, it also no longer makes sense to have the JavaScript implementation on
wasm32-unknown-unknown
be it's own crate. So we move the implementations back intogetrandom
proper, under the opt-in "js" feature.Issues to resolve:
"js"
feature that essentially says: "if I'm building onwasm32-unknown-unknown
assume I also have JavaScript"wasm32-unknown-unknown
and just assume that all users of that target are either usingwasm-bindgen
orcargo-web
.cargo doc --no-deps --open --package getrandom-js --target=wasm32-unknown-unknown
getrandom-js
name?@dhardy @newpavlov I can merge this if you don't have time to review, but let me know what you think and if you have any feedback.
@zer0x64 @Pauan @alexcrichton @koute I'd also like to have your thoughts on the above issues and which approach you would like best.