Explore location to generate SBOM precursor files

Similar to the generation of `depinfo` files, a function is called to
generated SBOM precursor file named `output_sbom`. It takes the
`BuildRunner` & the current `Unit`. The `sbom` flag can be specified as
a cargo build option, but it's currently not configured fully. To
test the generation the flag is set to `true`.

* use SBOM types to serialize data

Output source, profile & dependencies

Trying to fetch all dependencies

This ignores dependencies for custom build scripts. The output should be
similar to what `cargo tree` reports.

Output package dependencies

This is similar to what the `cargo metadata` command outputs.

Extract logic to fetch sbom output files

This extracts the logic to get the list of SBOM output file paths into
its own function in `BuildRunner` for a given Unit.

Add test file to check sbom output

* add test to check project with bin & lib
* extract sbom config into helper function

Add build type to dependency

Add test to read JSON

Still needs to check output.

Guard sbom logic behind unstable feature

Add test with custom build script

Integrate review feedback

* disable `sbom` config when `-Zsbom` is not passed as unstable option
* refactor tests
* add test

Expand end-to-end tests

This expands the tests to reflect end-to-end tests by comparing the
generated JSON output files with expected strings.

* add test helper to compare actual & expected JSON content
* refactor setup of packages in test

Add 'sbom' section to unstable features doc

Append SBOM file suffix instead of replacing

Instead of replacing the file extension, the `.cargo-sbom.json` suffix
is appended to the output file. This is to keep existing file extensions
in place.

* refactor logic to set `sbom` property from build config
* expand build script related test to check JSON output

Integrate review feedback

* use `PackageIdSpec` instead of only `PackageId` in SBOM output
* change `version` of a dependency to `Option<Version>`
* output `Vec<CrateType>` instead of only the first found crate type
* output rustc workspace wrapper
* update 'warning' string in test using `[WARNING]`
* use `serde_json::to_writer` to serialize SBOM
* set sbom suffix in tests explicitely, instead of using `with_extension`

Output additional fields to JSON

In case a unit's profile differs from the profile information on root
level, it's added to the package information to the JSON output.

The verbose output for `rustc -vV` is also written to the `rustc` field
in the SBOM.

* rename `fetch_packages` to `collect_packages`
* update JSON in tests to include profile information

Add test to check multiple crate types

Add test to check artifact name conflict

Use SbomProfile to wrap Profile type

This adds the `SbomProfile` to convert the existing `Profile` into, to
expose relevant fields. For now it removes the `strip` field, while
serializing all other fields. It should keep the output consistent, even
when fields in the `Profile` change, e.g. new field added.

Document package profile

* only export `profile` field in case it differs from root profile

Add test to check different features

The added test uses a crate with multiple features. The main crate uses
the dependency in the normal build & the custom build script with
different features.

Refactor storing of package dependencies

All dependencies for a package are indices into the `packages` list now.
This sets the correct association between a dependency & its associated
package.

* remove `SbomDependency` struct

Refactor tests to use snapbox

Author: arlosi

crates/cargo-test-support/src/lib.rs

@@ -415,6 +415,16 @@ impl Project {
             .join(paths::get_lib_filename(name, kind))
     }
 
+    /// Path to a dynamic library.
+    /// ex: `/path/to/cargo/target/cit/t0/foo/target/debug/examples/libex.dylib`
+    pub fn dylib(&self, name: &str) -> PathBuf {
+        self.target_debug_dir().join(format!(
+            "{}{name}{}",
+            env::consts::DLL_PREFIX,
+            env::consts::DLL_SUFFIX
+        ))
+    }
+
     /// Path to a debug binary.
     ///
     /// ex: `$CARGO_TARGET_TMPDIR/cit/t0/foo/target/debug/foo`

src/cargo/core/compiler/build_config.rs

@@ -48,6 +48,8 @@ pub struct BuildConfig {
     pub future_incompat_report: bool,
     /// Which kinds of build timings to output (empty if none).
     pub timing_outputs: Vec<TimingOutput>,
+    /// Output SBOM precursor files.
+    pub sbom: bool,
 }
 
 fn default_parallelism() -> CargoResult<u32> {
@@ -99,6 +101,17 @@ impl BuildConfig {
             },
         };
 
+        // If sbom flag is set, it requires the unstable feature
+        let sbom = match (cfg.sbom, gctx.cli_unstable().sbom) {
+            (Some(sbom), true) => sbom,
+            (Some(_), false) => {
+                gctx.shell()
+                    .warn("ignoring 'sbom' config, pass `-Zsbom` to enable it")?;
+                false
+            }
+            (None, _) => false,
+        };
+
         Ok(BuildConfig {
             requested_kinds,
             jobs,
@@ -115,6 +128,7 @@ impl BuildConfig {
             export_dir: None,
             future_incompat_report: false,
             timing_outputs: Vec::new(),
+            sbom,
         })
     }
 

src/cargo/core/compiler/build_runner/mod.rs

@@ -1,6 +1,7 @@
 //! [`BuildRunner`] is the mutable state used during the build process.
 
 use std::collections::{BTreeSet, HashMap, HashSet};
+use std::io::BufWriter;
 use std::path::{Path, PathBuf};
 use std::sync::{Arc, Mutex};
 
@@ -10,6 +11,7 @@ use crate::core::PackageId;
 use crate::util::cache_lock::CacheLockMode;
 use crate::util::errors::CargoResult;
 use anyhow::{bail, Context as _};
+use cargo_util::paths;
 use filetime::FileTime;
 use itertools::Itertools;
 use jobserver::Client;
@@ -291,6 +293,14 @@ impl<'a, 'gctx> BuildRunner<'a, 'gctx> {
             }
 
             super::output_depinfo(&mut self, unit)?;
+
+            if self.bcx.build_config.sbom {
+                let sbom = super::build_sbom(&mut self, unit)?;
+                for sbom_output_file in self.sbom_output_files(unit)? {
+                    let outfile = BufWriter::new(paths::create(sbom_output_file)?);
+                    serde_json::to_writer(outfile, &sbom)?;
+                }
+            }
         }
 
         for (script_meta, output) in self.build_script_outputs.lock().unwrap().iter() {
@@ -446,6 +456,33 @@ impl<'a, 'gctx> BuildRunner<'a, 'gctx> {
         self.files().metadata(unit).unit_id()
     }
 
+    /// Returns the list of SBOM output file paths for a given [`Unit`].
+    pub fn sbom_output_files(&self, unit: &Unit) -> CargoResult<Vec<PathBuf>> {
+        let mut sbom_files = Vec::new();
+        if !self.bcx.build_config.sbom || !self.bcx.gctx.cli_unstable().sbom {
+            return Ok(sbom_files);
+        }
+        for output in self.outputs(unit)?.iter() {
+            if matches!(output.flavor, FileFlavor::Normal | FileFlavor::Linkable) {
+                if let Some(path) = &output.hardlink {
+                    sbom_files.push(Self::append_sbom_suffix(path));
+                }
+                if let Some(path) = &output.export_path {
+                    sbom_files.push(Self::append_sbom_suffix(path));
+                }
+            }
+        }
+        Ok(sbom_files)
+    }
+
+    /// Append the SBOM suffix to the file name.
+    fn append_sbom_suffix(link: &PathBuf) -> PathBuf {
+        const SBOM_FILE_EXTENSION: &str = ".cargo-sbom.json";
+        let mut link_buf = link.clone().into_os_string();
+        link_buf.push(SBOM_FILE_EXTENSION);
+        PathBuf::from(link_buf)
+    }
+
     pub fn is_primary_package(&self, unit: &Unit) -> bool {
         self.primary_packages.contains(&unit.pkg.package_id())
     }

src/cargo/core/compiler/mod.rs

@@ -47,6 +47,7 @@ pub(crate) mod layout;
 mod links;
 mod lto;
 mod output_depinfo;
+mod output_sbom;
 pub mod rustdoc;
 pub mod standard_lib;
 mod timings;
@@ -60,7 +61,7 @@ use std::env;
 use std::ffi::{OsStr, OsString};
 use std::fmt::Display;
 use std::fs::{self, File};
-use std::io::{BufRead, Write};
+use std::io::{BufRead, BufWriter, Write};
 use std::path::{Path, PathBuf};
 use std::sync::Arc;
 
@@ -85,6 +86,7 @@ use self::job_queue::{Job, JobQueue, JobState, Work};
 pub(crate) use self::layout::Layout;
 pub use self::lto::Lto;
 use self::output_depinfo::output_depinfo;
+use self::output_sbom::build_sbom;
 use self::unit_graph::UnitDep;
 use crate::core::compiler::future_incompat::FutureIncompatReport;
 pub use crate::core::compiler::unit::{Unit, UnitInterner};
@@ -307,6 +309,8 @@ fn rustc(
     let script_metadata = build_runner.find_build_script_metadata(unit);
     let is_local = unit.is_local();
     let artifact = unit.artifact;
+    let sbom_files = build_runner.sbom_output_files(unit)?;
+    let sbom = build_sbom(build_runner, unit)?;
 
     let hide_diagnostics_for_scrape_unit = build_runner.bcx.unit_can_fail_for_docscraping(unit)
         && !matches!(
@@ -392,6 +396,12 @@ fn rustc(
         if build_plan {
             state.build_plan(buildkey, rustc.clone(), outputs.clone());
         } else {
+            for file in sbom_files {
+                tracing::debug!("writing sbom to {}", file.display());
+                let outfile = BufWriter::new(paths::create(&file)?);
+                serde_json::to_writer(outfile, &sbom)?;
+            }
+
             let result = exec
                 .exec(
                     &rustc,
@@ -685,6 +695,7 @@ where
 /// completion of other units will be added later in runtime, such as flags
 /// from build scripts.
 fn prepare_rustc(build_runner: &BuildRunner<'_, '_>, unit: &Unit) -> CargoResult<ProcessBuilder> {
+    let gctx = build_runner.bcx.gctx;
     let is_primary = build_runner.is_primary_package(unit);
     let is_workspace = build_runner.bcx.ws.is_member(&unit.pkg);
 
@@ -700,7 +711,7 @@ fn prepare_rustc(build_runner: &BuildRunner<'_, '_>, unit: &Unit) -> CargoResult
         base.args(args);
     }
     base.args(&unit.rustflags);
-    if build_runner.bcx.gctx.cli_unstable().binary_dep_depinfo {
+    if gctx.cli_unstable().binary_dep_depinfo {
         base.arg("-Z").arg("binary-dep-depinfo");
     }
     if build_runner.bcx.gctx.cli_unstable().checksum_freshness {
@@ -709,6 +720,8 @@ fn prepare_rustc(build_runner: &BuildRunner<'_, '_>, unit: &Unit) -> CargoResult
 
     if is_primary {
         base.env("CARGO_PRIMARY_PACKAGE", "1");
+        let file_list = std::env::join_paths(build_runner.sbom_output_files(unit)?)?;
+        base.env("CARGO_SBOM_PATH", file_list);
     }
 
     if unit.target.is_test() || unit.target.is_bench() {