DEV: workflow with the fingerprinting / hash in index.css/js filenames

[jorisvandenbossche]

Recently, we started adding a hash (fingerprint) to the index.css and index.js filenames:

https://github.com/pandas-dev/pydata-sphinx-theme/blob/df856900e829d70307c5e93bc20a153eddae5187/webpack.common.js#L103

This is useful (good practice, and actually also solved an issue we had in pandas with cached assets on the server when releasing a new version), but, makes the development process also more inconvenient ..

Because now each time you make a change in the css/js source code, and the static assets thus get recompiled, the filename of pydata_sphinx_theme/static/css/index.[hash].css changes. That means that you need to "git remove" the old file (which is automatically deleted on the filesystem, but you still need to tell git this is OK) and "git add" the new each time you want to commit those changes.

So I am wondering if we can make this workflow a bit smoother. Some ideas:

• I suppose the hash is currently based on the full src/ content. But I was wondering: would it be possible to let this hash be defined on the package version number? That would still ensure that for people using released versions they always get a different hash when updating the pydata-sphinx-theme version, and only people relying directly on master might need to be careful about caching of old assets.
  From a quick google, this doesn't seem to be an option with webpack.
• Alternatively, it might be possible to let yarn build:production/dev do the git rm / git add itself?

Or do you have other experience with how to deal with such workflow?

cc @bollwyvl @hoetmaaiers

[bollwyvl]

I would think `git add src/static` just does the right thing? I advise against automating git activities, usually. Using the version number will make weird cache behavior more likely if it's in fact true people run off master, this will defeat the fingerprinting if something does change.

[hoetmaaiers]

I agree with @bollwyvl, the hash should best resemble the actual code it represents. This way a minimal change (even a typo) will bust the cache. Doesn't git add src/static work?

[jorisvandenbossche]

I would think git add src/static just does the right thing?

Ah, yes, you're right. I didn't actually try that (in the assumption I needed to do git rm pydata_sphinx_theme/static/css/index<hash>.css manually), but indeed, doing a git add pydata_sphinx_theme/static both directly removes the old and adds the new css files to the git tracked files.

OK, that's certainly already more manageable. I will do an update of contriburing docs to add a comment about this, then.
Thanks both for your comments!

[jorisvandenbossche]

Another workflow issue that has come up in a related repo about this: with the hash changing for each commit (that edits CSS / JS), that also means that if there are multiple PRs open, all PRs will have conflicts and need to be updated once one other PR is merged.
See executablebooks/sphinx-panels#32 (comment) and comments below that for some related discussion.

Now, the solution for this would basically be to not include the compiled CSS, and only compile during a build step (like if you have a C extension), meaning you can't directly use a git checkout / install from master anymore ..

[pradyunsg]

meaning you can't directly use a git checkout / install from master anymore ..

You can -- if you use a compile toolchain that's pip-installable. Given that libsass-python exists and provides wheels for basically every major platform, that's definitely possible if all you've got is SCSS code!

[pradyunsg]

Given that libsass-python exists and provides wheels for basically every major platform, that's definitely possible!

And, if you want to use a more modern JS tooling, then... well... you'll need to require the user to have npm installed, and could run an "npm install" followed by "gulp build" or whatever during the build steps for the wheel build -- essentially requiring npm instead of gcc. :P

[choldgraf]

Another option we could follow here is to try digest= parameters in our HTML templates. I think that this will allow us to bust the cache without hard-coding it in the filename. This has worked nicely in the book theme and allows others to sub-theme more easily as well.

the idea is that in the linked HTML the CSS is loaded like:

<link ... src="myfile.css?digest=[SOMEHASH] />

and you just need to update the hash in the HTML when you re-compile assets, but not in the CSS file itself. I'm not sure if this works with JS assets or not, but I think it does because this is an HTML thing not a CSS thing per-se

[choldgraf]

I believe that this one is now resolved by #514