Add buildWorkspace

[sellout]

I’m not sure if this is a good idea, but if not I’d like to hear what the best alternative is (explicit separate buildPackage for each lib in the workspace?)

In any case, I want roughly

  outputs = { self, crane, flake-utils, nixpkgs, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        craneLib = crane.lib.${system};
        pkgs = import nixpkgs {inherit system;};
      in
    {
      packages =
        {default = self.packages.${system}.oneOfTheOnesInTheWorkspace;}
        // craneLib.buildWorkspace {
          src = craneLib.cleanCargoSource ./.;
          doCheck = true;
          buildInputs = [pkgs.libiconv];
          nativeBuildInputs = [];
        };
    });

where buildWorkspace produces an attrset of packages, one per package in the workspace. And the attrs passed to buildWorkspace are propagated to the individual packages. Although maybe it makes sense to put propagated attrs in a single attr, so other buildWorkspace attrs (like, maybe an exclude list or something) can be provided without conflicting or being propagated.

I’m still on the steep part of the learning curve for both Nix and Rust, so I’m as interested in arguments against this as I am in seeing it done.

[ipetkov]

Hi @sellout thanks for the report! Let me ask a bit more concretely: what potential advantages would you be interested in seeing frombuildWorkspace which would be impossible with buildPackage? (keeping in mind that buildPackage can easily build all binaries in the workspace 😉 )

---

From the perspective of organizing derivations and their outputs I could see the advantage of being able to select a specific "output" of the workspace (I know crate2nix supports this). That said, I see a couple of reasons why I wouldn't consider this direction (alone) a motivator:

• implementation costs: this would require even deeper analysis of the project structure to figure out what the outputs are (I don't know of any cargo-related tools which allow for easy/arbitrary queries of the project structure). With build-scripts I have a feeling it becomes an undecidable problem. Even if figuring out all the outputs can be solved (elegantly?) there is still the issue of knowing which targets to actually build. Some targets may depend on others, but having separate outputs is kind of moot if you have to build the whole workspace as the granularity doesn't afford you much of anything (besides maybe a derivation with one /bin/* output)
• At the end of the day, each project maintainer understands their project structure the best. if they want to build a subset of the workspace I would recommend setting the right cargo flags as to avoid unnecessary work. If they wish to build the entire workspace but split up the outputs into separate derivation outputs (for organization's sake) it is pretty trivial to do with some runCommand invocations.

---

Where i do see a stronger motivator for buildWorkspace is more efficient incremental caching within the workspace itself. For example, consider a workspace with a large number of crates where the "leaf" crates receive the most amount of changes and the "root" crates receiving much fewer changes. It would be really nice to get incremental caching such that we don't have to rebuild the entire workspace when only the "leaf" crates change.

I've had some ideas for implementing something like this. I think the basic approach would be using something like cargo-guppy to get a topological sort of the workspace's crates. We can then build a chain of buildPackage derivations each depending on the cargoArtifacts of the previous derivation, as well as filtering the source files to only contain the current crate and its dependencies' sources to avoid invalidation.

[sellout]

So, I could be off the mark here, but here’s what I’m thinking. First, I mostly build libraries, so the flake is largely for developer/CI consistency and efficiency. So yeah, my use case is basically what you state in the second part. I do something similar for other projects. Here is a Haskell example, with less automation than Crane provides: https://github.com/con-kitty/concat/blob/506a9461079fa407df4b81f4d1bb00f84671e7f0/flake.nix

Line 30 processes the “cabal.project” file, which has a packages section somewhat similar to a [workspace] stanza. It then uses that for a few things:

• line 36: produce an overlay with all the packages in the right place;
• line 54: produce a set of packages (cartesian product of compiler versions and individual packages, plus one with the full set of packages loaded withPackages style); and
• line 58: produce a set of devShells (one for each compiler version we care about) with the dependencies for all the packages in the “workspace”.

An extension of the incremental build/caching is that the separate packages are great to have for garnix to get fine-grained CI failure reporting.

About the first part – you’re right, that’s a tough and frustrating problem. Our Haskell solution is hacky, and I was thinking Crane’s could be at least less hacky (but I’m only guessing, I don’t know cargo tooling at all). It sounds like cargo-guppy could do one better than a topological sort, and give you the exact set of direct dependencies that intersect with workspace.members to depend on (and depending on only the -deps package if the intersection is empty.

In Haskell we just depend on the overlays.default, which contains all of the packages from the “workspace”, so each entry in packages doesn’t know its explicit dependencies, but can find any of them via te overlay. Less precise, but easy.

[dpc]

It might be related and/or helpful. In the project I'm using crane for heavily, we have both package-based outputs, and workspace-based outputs. Workspace ones are used for CI, where we quickly and robustly try to build/test everything in aggregate. Packages are useful for OCI containers, or when building things for architectures (like WASM) where not all the code can even compile. Notably, in package builds, we also list all the directories that are actually being used, to help the caching and avoiding rebuild (at the cost of having to manually maintain these lists).

[IreneKnapp]

hi! I came here to say that I found this explanation for why the feature doesn't exist (yet) to be useful and think it should make its way into the documentation. I think that officially documenting the reasoning here would be worthwhile even if the feature is ultimately added, since it would clarify the relationship between the two functions.

[cognivore]

I would like to chime in.

As a very hands-on person, who wants to solve business problems, I take to heart the "easily" quantifier in

keeping in mind that buildPackage can easily build all binaries in the workspace 😉

Suppose I have a multi-crate workspace. No tricks, including the one that @ipetkov has described in a similar issue have saved me from

error: builder for '/nix/store/qwghrad7sgcrbjgwx9wj0zv978dvfgq9-rootrunner-deps-0.1.0.drv' failed with exit code 101;
       last 22 log lines:
       > cargoArtifacts not set, will not reuse any cargo artifacts
       > Running phase: unpackPhase
       > unpacking source archive /nix/store/rmjiyggq415r8llm1h5i7g4x9vnb8sji-source
       > source root is source
       > Running phase: patchPhase
       > Executing configureCargoCommonVars
       > Running phase: updateAutotoolsGnuConfigScriptsPhase
       > Running phase: configurePhase
       > will append /build/source/.cargo-home/config.toml with contents of /nix/store/csafzlpfnyjdpx3i4bqs5f7j7a3s6khc-vendor-cargo-deps/config.toml
       > default configurePhase, nothing to do
       > Running phase: buildPhase
       > ++ command cargo --version
       > cargo 1.82.0 (8f40fc59f 2024-08-21)
       > ++ command cargo check --release --bin rootrunner-exe --all-targets
       > error: failed to load manifest for workspace member `/build/source/crates/*`
       > referenced by workspace at `/build/source/Cargo.toml`
       >
       > Caused by:
       >   failed to read `/build/source/crates/*/Cargo.toml`
       >
       > Caused by:
       >   No such file or directory (os error 2)
       For full logs, run 'nix log /nix/store/qwghrad7sgcrbjgwx9wj0zv978dvfgq9-rootrunner-deps-0.1.0.drv'.
error: 1 dependencies of derivation '/nix/store/8qaf1nx2ibx3gqqfc4gnrpp78nbgqsks-rootrunner-0.1.0.drv' failed to build

My situation is a bit different though, I use ./crates/xyzzy as a library in the main crate, so perhaps there is some complication when the dependencies go from workspace members to the root crate?.. Although one would presume that it's quite a frequently-occurring scenario.

This is not a call to action at all. This is simply a user sharing their experience. I have an immense respect for crane's author and contributors and use it for simpler / smaller projects.

[Smona]

I have a similar use case to what @ipetkov described above - a workspace with a leaf bin crate that changes very frequently, and lib crates that change much less frequently. I had some issues with the recommended approach for workspaces:

• I didn't want buildDepsOnly to rebuild on any workspace code changes.
• I didn't want to use cargo-hakari or a workspace hack package. I didn't want to include any features in a workspace crate that it doesn't actually need.
• I wanted to minimize cache misses from the cargoArtifacts in each crate build.
• I didn't want the shared dependency build to include native dependencies (C libraries, pkg-config) for all of the crates in the workspace in its closure.

The key change for me was building separate cargoArtifacts for each workspace crate. While this increases the total amount of compilation that has to happen for each change to a Cargo.toml (you're re-building n dependency environments when any of the Cargo.tomls change), it keeps the individual buildDepsOnly calls much smaller, and they can be parallelized. Since most changes are code-only with no dependency changes, it feels like the right tradeoff to me.

I ended up creating a helper function:

buildWorkspaceCrate helper function

{ pkgs, craneLib }:
let
  # Minimal workspace sources needed for Cargo to understand the workspace structure.
  # Changes to these files will rebuild all crate cargoArtifacts.
  # All crates in the workspace must have their Cargo.tomls in this list.
  workspaceStubFileset = pkgs.lib.fileset.unions [
    ./Cargo.toml
    ./Cargo.lock
    # Include only Cargo.toml file for each workspace, requires that they all have
    # explicit [lib] and/or [[bin]] sections since the source code
    # won't be there for cargo to infer the crate type.
    ./binary_crate/Cargo.toml
    ./lib_a/Cargo.toml
    ./lib_b/Cargo.toml
  ];
in
{
  # Convenience function for building a workspace crate with crane+nix.
  buildWorkspaceCrate =
    {
      # The crate root directory, as a path.
      src,
      # By default non-rust sources are filtered out of the build environment.
      # Specify any non-rust sources needed to build the crate here.
      nonRustSrcs ? [ ],
      # Paths to all workspace crates that this crate depends on.
      workspaceDeps ? [ ],
      buildInputs ? [ ],
      nativeBuildInputs ? [ ],
      ...
    }@crateArgs:
    let
      meta = craneLib.crateNameFromCargoToml { inherit src; };
      # Crane arguments shared between cargoArtifacts and the crate itself.
      commonArgs = {
        strictDeps = true;
        inherit buildInputs nativeBuildInputs;
      };

      finalSrc = pkgs.lib.fileset.toSource {
        root = ./.;
        fileset = pkgs.lib.fileset.unions (
          [
            workspaceStubFileset
            (craneLib.fileset.commonCargoSources src)
          ]
          ++ (builtins.map (path: (craneLib.fileset.commonCargoSources path)) workspaceDeps)
          ++ nonRustSrcs
        );
      };
      # Build dependencies specific to this package
      packageCargoArtifacts = craneLib.buildDepsOnly (
        commonArgs
        // {
          inherit (meta) pname version;
          src = finalSrc;
          cargoExtraArgs = "-p ${meta.pname}";
        }
      );
    in
    craneLib.buildPackage (
      commonArgs
      // meta
      // crateArgs
      // {
        src = finalSrc;
        cargoExtraArgs = "-p ${meta.pname}";
        cargoArtifacts = packageCargoArtifacts;
      }
    );
}

The per-crate derivations look something like this:

# binary_crate/default.nix

{ pkgs }:

let
  workspace = import ../.;
in
workspace.buildPackage {
  src = ./.;
  workspaceDeps = [ ../crate_a ../crate_b ];
  nonRustSrcs = [
    ./migrations
    ./src/graphql/documents
  ];

  buildInputs = with pkgs; [ openssl ];
  nativeBuildInputs = with pkgs; [ pkg-config ];

  # Additional environment variables or build phases/hooks can be set
  # here *without* rebuilding all dependency crates
  GRAPHQL_SCHEMA = ../../web_app/schema.gql;
}

I just got this working so it remains to be seen whether it will hold up well in the long-run, but I've tested it via changes to different parts of the workspace tree, and the caching is working perfectly for me, resolving all of my concerns above. Hopefully it can be of use to anyone who's in a similar situation I am of wanting to efficiently build minimal closures of individual crates in a workspace.