Reduce JS bundle size

[v6ak]

I have a static site generated by Scala.js + sbt-web + HTML generators. It currently uses Play framework for development (not for production). I have tried to replace Play by Vite in order to speed up development and get a newer SASS compiler. The main issue is bundle size, which increased from ~1MiB to ~1.9MiB.

I know this is related to ES modules (required by Vite), which cause the Google Closure Compiler to be disabled. Related project-specific issue: v6ak/zbdb-stats#57

[Lukah0173]

There's some further info in the Scala.js repo:

scala-js/scala-js#4482
scala-js/scala-js#3893

It seems to be a few people considering / waiting on a native solution in the Scala.js toolchain. In the meantime, it's not ideal, but it's possible to use Terser to minimise assets (tested with Vite 4.4.X). Although, be cautious if dynamic imports are used as described in the above issue threads.

build: {
    ...
    minify: 'terser',
    terserOptions: {
        sourceMap: true,
        nameCache: {},
        format: {
            comments: false,
        },
        mangle: {
            properties: {
                debug: false,
                keep_quoted: true,
                reserved: ['$classData', 'main', 'toString', 'constructor', 'length', 'call', 'apply', 'NaN', 'Infinity', 'undefined'],
                regex: /^(Lweb|Lslinky|slinky|render__L|query__L|\$m_|.*__f_|Ljava|cats\$)/,
            }
        }
    },
},

[v6ak]

@Lukah0173 Thanks for linking the relevant issues.

Terser looks cool, I'll probably try it. Does it affect 3rd party JS (non-Scala.js) libraries? I think they deserve a more conservative approach (unless we are sure we can perform some more aggressive optimizations).

[Lukah0173]

Yep, it should also work for the third-party libraries - I'm not sure about external JS that haven't been transpiled by Scala.js. I've tested with the following and no issues, but I don't know enough about the concepts to be certain of safety / effectiveness.

We're using the following dependencies:

      Compile / npmDependencies ++=
        List(
          "@types/d3" -> "7.4.0",
          "@ui5/webcomponents" -> "1.17.0",
          "@ui5/webcomponents-fiori" -> "1.17.0",
          "@ui5/webcomponents-icons" -> "1.17.0",
          "d3" -> "7.4.0",
          "date-fns" -> "2.29.3",
          "date-fns-tz" -> "2.0.0",
          "keycloak-js" -> "19.0.2",
          "simple-statistics" -> "7.7.6",
          "tailwindcss" -> "3.3.0",
          "typescript" -> "4.6.2",
        ))

and, in addition to our source, here's the result of the build:

vite v4.4.11 building for production...
✓ 735 modules transformed.
../dist/index.html                     0.79 kB │ gzip:   0.47 kB
../dist/assets/index-51df973b.css     37.61 kB │ gzip:   6.90 kB
../dist/assets/app-a95d1b34.js     4,375.23 kB │ gzip: 749.36 kB

It only takes a few seconds or so to build, ymmv.

[v6ak]

I have similar list of JS dependencies (bootstrap, chart.js, moment +tz, chartjs-adapter-moment and comma-separated-values). The resulting size is much better, but still by ~330KiB larger than it is with closure compiler:

dist/assets/index-1a9a2c3d.js  1,389.92 kB │ gzip: 293.20 kB │ map: 5,549.54 kB

Also, it doesn't like the moment's style of calling functions:

client/target/scala-3.3.1/zbdb-stats-client-opt/client.js (8132:15) Cannot call a namespace ("$i_moment").

For now, I'll probably use Vite for dev only, keeping sbt-web for production.

[gzm0]

Also, it doesn't like the moment's style of calling functions:

That looks like there is a namespace import in a facade instead of a default import.

[v6ak]

I still believe there must be a way to proceed with Google Closure Compiler, even with few hacks. I believe I've almost reached the result, albeit with some hacks:

1. Produce ES modules with Scala.js
2. Run postprocessing and Closure Compiler: https://gist.github.com/v6ak/ccd0cadb43993afa854769519646ed46
3. Manually rewrite opt.js: replace require with imports. (I don't have a script for that.)
4. Build it with Vite

It seems to produce a sane result with the exception of calls to 3rd-party code. Calls to module's top-level functions seem to be OK, but calls to methods of their objects are mangled.

I've looked how Scala.js configures the Closure Compiler. It seems that the difference is primarily in the externs. It however seems that I cannot easily export them, as they seem to be kept in-memory. But maybe, when I adjust the code, I can get the externs.

[sjrd]

The externs in a tiny part of it. What really matters is that we emit external method/property references as foo["bar"] instead of foo.bar.

[v6ak]

Makes sense, but I wonder why it works with CommonJS modules. IIUC, there should be the same issue.

[sjrd]

For CommonJS, GCC leaves the require(...) calls alone, because they're normal, external function calls. It doesn't want to leave import statements alone, however.

[v6ak]

In my experiment, I've replaced all external JS imports (i.e., those that don't start with "./"*) by require function call before passing to GoCC, so the reason shouldn't be there. The difference might be in the type annotation of the require function. Maybe I need some recursive typedef like Dynamic = Object<String, Dynamic>. *) Well, my hacky regex checks just for the initial dot.

[v6ak]

Aha, I got it. Scala.js uses withOptimizeBracketSelect(false) when using GoCC, so GoCC doesn't mangle the references. With ES module, there is no way to configure optimizeBracketSelect, so GoCC mangles it.

I can play a bit more with that, and hopefully even send a PR. However, I probably can't get rid of the hack (rewriting imports to function calls before passing to GCC and rewriting it back when GCC is done). I just can write it in somewhat cleaner way (using AST transformations instead of regex, using a dedicated scalajs-specific name rather than require). Is this hack acceptable?

[sjrd]

Is this hack acceptable?

Probably not. But if it allows us to reliably use GCC with ES modules, we can at least consider it.

[v6ak]

I hope I can make GoCC reliably working with ES modules. It should work even for multi-module projects (internal modules are inlined). Inlining seems to be a suitable option for many scenarios (server/client/webworker code).

However, my willingness to work on it depends on your willingness to accept such patch. If you say that this hack would be probably unacceptable, I don't want to spend much time on that.

[sjrd]

Inlining seems to be a suitable option for many scenarios (server/client/webworker code).

Well ... it shouldn't contradict what the ModuleSplitStyle of the Scala.js guarantees. In particular in the presence of dynamic js.import() calls.

[v6ak]

Ad ModuleSplitStyle: OK, it sounds like some redesign would be needed for what I want. I'd like to use small modules for fast link and fat modules (=inlined internal modules) for production.

I have some good and bad news:

The bad: I struggle with AST rewriting.

The good: I've mostly succeeded by going some hacky way. It seems that I need to slightly adjust imports of moment.js and BS Modal, but everything else is probably fine. (EDIT: I've done few more adjustments and it works 100%! Also, the bundle size roughly matches the original non-Vite bundle size.) How I did it:

1. I adjusted the SBT plugin for Scala.js: added .withOptimizeBracketSelects(false) to BasicLinkerBackend. I can make it configurable and send a PR.
2. Compiled multiple modules with scalaJSLinkerConfig ~= {_.withModuleKind(ModuleKind.ESModule).withModuleSplitStyle(ModuleSplitStyle.FewestModules)}.
3. Replaced imports by requires (just 3rd party libraries; imports of internal modules were kept).
4. Ran google Closure Compiler.
5. Replaced requires by imports.
6. Ran Vite build

Updated script (it currently does both adjustments automatically): https://gist.github.com/v6ak/ccd0cadb43993afa854769519646ed46