Skip to content

Support outputting wasm as inline base64 #1880

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

samdenty
Copy link

@samdenty samdenty commented Nov 26, 2019

It's difficult to use wasm-bindgen in libraries meant to be consumed across a variety of bundler setups.

Given an application using pkg-a -> pkg-wasm, you'd need to ship separate builds for both pkg-wasm and pkg-a. This will be fixed when .wasm imports become mainstream.

In the meantime, this PR adds support for an inline option that outputs the wasm as Base64, in the JS file.

This alleviates the need for separate builds, at the cost of a slightly larger bundle / no async streaming compiler.

Expected JS output
const wasm = (() => {
    const imports = {};
    imports.wbg = {};
    imports.wbg._abc = ...

    const raw = atob('...')
    const rawLength = raw.length
    const buf = new Uint8Array(new ArrayBuffer(rawLength))
    for(var i = 0; i < rawLength; i++) {
    buf[i] = raw.charCodeAt(i)
    }

    const mod = new WebAssembly.Module(buf)
    return new WebAssembly.Instance(mod, imports).exports
})()

export function greet() {
    wasm.greet();
}

let cachedTextDecoder = new TextDecoder('utf-8', { ignoreBOM: true, fatal: true });

cachedTextDecoder.decode();

let cachegetUint8Memory = null;
function getUint8Memory() {
    if (cachegetUint8Memory === null || cachegetUint8Memory.buffer !== wasm.memory.buffer) {
        cachegetUint8Memory = new Uint8Array(wasm.memory.buffer);
    }
    return cachegetUint8Memory;
}

function getStringFromWasm(ptr, len) {
    return cachedTextDecoder.decode(getUint8Memory().subarray(ptr, ptr + len));
}

@samdenty
Copy link
Author

samdenty commented Nov 26, 2019

I searched around, but I'm not sure I found where the --inline flag is passed to wasm-bindgen. Link to it would be helpful

@samdenty samdenty marked this pull request as ready for review November 26, 2019 21:07
@Pauan
Copy link
Contributor

Pauan commented Nov 26, 2019

To my knowledge, it should be possible to use the bundler target with all the major bundlers (Webpack, Parcel, and Rollup). Are you having issues with that?

(Though this is probably an interesting idea regardless).

@samdenty
Copy link
Author

samdenty commented Nov 26, 2019

@Pauan Though most tools do support it, there's a few that don't facebook/create-react-app#4912 / node requiring extra flag

It's copied from rollup, which does it with @rollup/plugin-wasm for wasm smaller than 4kb

Although not ideal, it's intended for library authors.

@Pauan
Copy link
Contributor

Pauan commented Nov 27, 2019

@samdenty Ah, right, I forgot about things like Angular (and apparently React) disabling wasm support (even though they use Webpack internally, which does support wasm out of the box).

@alexcrichton
Copy link
Contributor

Thanks for the PR! I do agree with @Pauan though that this seems best left to bundlers rather than a different emission mode of wasm-bindgen itself. (we could add tons of various emission options but in general we try to leave most of them to bundlers). Can you speak more as to why, if webpack supports this, it's not supported through your use case? Is this perhaps best done as a feature request for those systems to pass through webpack configuration options?

@Pauan
Copy link
Contributor

Pauan commented Dec 2, 2019

@alexcrichton The problem is that some of those projects (e.g. Angular) have refused to add support for .wasm, even when asked. Their argument is basically, "we don't want to commit to anything, because the Wasm<->ESM spec hasn't been finalized". Although I disagree with them, they do have a point.

@alexcrichton
Copy link
Contributor

That's a bummer :(

If that's the case we could try to explore our own workaround, but I think there's a few things we'd want to consider:

  • The name --inline is pretty innocuous, I think we'll want to name it something else like --web-inline-base64 or something like that.
  • I don't think the usage of new here is quite right, this can run into the 4kb limit Chrome has, right?
  • Can book documentation be added for this flag, and when it might be used?
  • Can tests be added for this flag as well?

@alexcrichton
Copy link
Contributor

This hasn't seen movement in quite some time now so I'm going to close this, feel free to resubmit it though!

@D1plo1d
Copy link

D1plo1d commented Apr 9, 2020

I'm using Parcel 2 because of it greatly improves compile times but it also unfortunately lacks wasm support atm. I'd like --web-inline-base64 as a workaround for Parcel 2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants