GitHub tarballs aren't (guaranteed to be) reproducible

[kylewlacy]

GitHub offers source .tar.gz files of the tags of a repo. This is really convenient for Brioche, since we can just download and unpack them. Here's a snippet from the ripgrep package:

export const project = {
  name: "ripgrep",
  version: "14.1.0",
};

const source = Brioche.download(
  `https://github.com/BurntSushi/ripgrep/archive/refs/tags/${project.version}.tar.gz`,
)
  .unarchive("tar", "gzip")
  .peel();

The problem is, these tar files aren't guaranteed to be byte-for-byte identical. In the context of brioche-packages, this wouldn't necessarily cause an immediate breakage since the registry should save a copy of the downloaded recipes, effectively acting as an archive of the downloads we've used. But, outside the brioche-packages repo this can cause problems if GitHub ends up silently changing a .tar.gz and it has a hash that doesn't match the lockfile. Even within brioche-packages, if someone submits a new package (or an update to a package) with one hash in their lockfile, then it gets merged, then the build could fail if the hash changed!

We can alternatively use Brioche.gitRef() plus git.gitCheckout() to do something that's very similar, as can be seen in this old revision of the asciinema package:

export const project = {
  name: "asciinema",
  version: "2.4.0",
};

const source = gitCheckout(
  Brioche.gitRef({
    repository: "https://github.com/asciinema/asciinema.git",
    ref: `v${project.version}`,
  }),
);

This works very similarly, but instead of the lockfile recording the file hash of the potentially unstable .tar.gz file, it instead records the (hopefully stable) git commit hash. Even if the tag itself changed, the repository should hopefully still have the original commit we were referencing

---

I was inspired to open this Discussion from this PR: #171

[jaudiger]

When you say:

it instead records the (hopefully stable) git commit hash

Are you referring to the possibility of someone updating the tag's commit ? The commit of a tag should never be updated on public repository, since the tag has to be seen as a source of truth. But this operation is not forbidden. What does happen when someone is updating the recipe of a brioche package with the git repository tag referred in Brioche.gitRef moved to another commit ? Is the lock file updated ?

[kylewlacy]

Brioche doesn't really care if a tag (or a branch, for that matter) gets updated upstream. It works basically how Cargo does with git dependencies. When you add Brioche.gitRef(...) in a .bri file, Brioche will first check the lockfile. If the lockfile has the commit hash that we need, then we use that. If it doesn't, then we check the upstream repository, get the commit hash for the tag / branch, save it in the lockfile, and use that commit.

So the real issue would be if the commit disappeared: say, if someone force-pushed after amending a commit-- although GitHub usually keeps the old commit hash around, so even this isn't normally a problem.

What does happen when someone is updating the recipe of a brioche package with the git repository tag referred in Brioche.gitRef moved to another commit ? Is the lock file updated ?

As long as the lockfile has some commit for a specific branch/tag, we will always use that commit, and never check the upstream repo again. If the .bri file is updated, then of course it could point to a different tag that we then check and repeat the process. And eventually, there should be like a brioche update command which will let you manually update the lockfile. But for now, the only way to update a branch / tag is to delete the lockfile (or manually edit it)

[jaudiger]

Understood, thanks for the explanation.

We should then try to always reach the goal of a tool like brioche, which is simply the build and its reproducibility over time. If GitHub does not guarantee a byte to byte tarball on multiple download attempts (even if the probability is really low, and I think they have taken into account the errors from the past. So even if the blog post permits it, the case of updating a tarball on their side, for whatever the reason they had, should be pretty rare).

But then, let's take the assumption the tarball can change at any point, we should try to always rely on the most efficient solution to reach the reproducibility goal. The commit hash does serve this purpose, I guess, starting from now, if the package can be retrieved from a git repository, we should always use the commit hash to get the source code of the package. Meaning we should call both functions Brioche.gitRef and gitCheckout to retrieve the source code.

From the package maintainer point of view, would it be interesting to have a wrapper function to update the lock file with the correct commit hash and to checkout the git repository ? For example something like that:

const source = briocheGitFetch({
    repository: "https://github.com/brioche-dev/CMake.git",
    ref: "base/brioche-patches",
  });

[kylewlacy]

From the package maintainer point of view, would it be interesting to have a wrapper function to update the lock file with the correct commit hash and to checkout the git repository ?

Yep, would definitely love to support this! It's kind of a tall order though: Brioche.gitRef is a static, meaning it gets special "pre-processing" treatment by the Rust side. Specifically, Brioche.gitRef (as well as the other statics) must be called with a string literal, or a very specific type of string template (for more context, see brioche-dev/brioche#126). Because of this pre-processing step, we couldn't write a wrapper around Brioche.gitRef within brioche-packages directly (if you tried, you'd get an error about the static not being found).

When you call Brioche.gitRef, the actual JS code that runs lives within std/core/global.bri. Like the other static functions, the JS implemetation for Brioche.gitRef is fairly simple: it just calls the Rust side and asks "what commit do you have in the lockfile for this repo + tag?", then returns the value.

Okay, so that's the background info. If we want have a simplified gitCheckout function, I can think of two easy-ish options:

1. Add a new Brioche.gitCheckout static. The pre-processing step on the Rust side would treat it identically to Brioche.gitRef: it would look up the tag in the repo and save it in the lockfile. Then, for the JS side, we'd just do what gitCheckout does today. But most likely, we'd want to provide that function from std, meaning we'd need to get a build of git working within std including its dependencies.
2. Add a new Brioche.gitCheckout static. The pre-processing step on the Rust side would look up the tag and save it in the lockfile (just like Brioche.gitRef), then the Rust side would handle the checkout logic-- say, using gitoxide. For the JS side, we'd just retrieve the checkout the Rust side already did.

(1) is literally just combining the existing Brioche.gitRef + gitCheckout functions into one, but with some support from the runtime so it recognizes the new function name. It has the disadvantage that we'd be carrying around a build of git within std, which would mean we'd need to bring all of git's dependencies in as well (curl and openssl at least, and probably ca_certificates until brioche-dev/brioche#156 gets resolved).

(2) simplifies things on the runtime side since the heavy lifting is on the Rust side. But that's also the biggest drawback, since now Brioche itself needs to take care of checking out the repo. I don't think that'd be too bad, but then we need to address questions around submodules, git LFS, and shallow vs. deep clones within Brioche itself.

Between the two, I think (1) provides the least-painful option, but figuring out how to get a build of git working within std seems pretty daunting... (unless we somehow could put the Brioche.gitCheckout function into the git package instead of std?)

[jaudiger]

As usual, thanks for the explanation/clarification of the brioche codebase. It helps me to understand the decisions taken until today.

In fact, I was thinking about (3) as you begin to suggest here:

(unless we somehow could put the Brioche.gitCheckout function into the git package instead of std?)

Having a Brioche.gitCheckout function exposed by the git package could be a middle solution between (1) and (2) and it should be easy to address now. At least, until we have another suitable solution (if it's still needed one time).

Maintaining (2) inside the Brioche codebase does not bring any value for brioche itself, it'll be only here to resolve a problem inside the JS configuration file to help for the packaging. And having (1) as you already mentioned would be cumbersome.