Update rust-gpu to the latest main

[nazar-pc]

Simply updates to the latest main, otherwise patching is quite cumbersome with git dependencies.

I'm specifically interested in pulling Rust-GPU/rust-gpu#302.

[tombh]

I see you're interested in that specific feature in Rust-GPU/rust-gpu#302. So I just wanted to note that that change is in rust-gpu's rustc-codegen-spirv crate. Whereas this PR is updating rust-gpu's spirv-builder crate.

The way cargo-gpu works is that it uses the rustc-codegen-spirv version (via spirv-std) as defined in the user's shader crate. In fact, cargo-gpu doesn't specify any particular spirv-std version, its goal is to actually work with all spirv-std versions from the last 1-2 years.

So if your intention is to just compile your shader with a newer version of rust-gpu then you need to update the dependencies.spirv-std.version in your shader's Cargo.toml. Here's an example of that line in our shader tempalate.

I think if that's not clear from our README and docs we should update them.

But of course you may also want to update spirv-builder too! Which is great. But I'm not sure there have been any changes there recently?

[nazar-pc]

Hm... what you said doesn't quite make sense to me.

spirv-builder depends on rustc_codegen_spirv crate directly, spirv-std has nothing to do with that directly. And code changes in Rust-GPU/rust-gpu#302 are in the codegen backend. Am I missing something?

[tombh]

Right, we could be talking about different things. Apologies. And also re-reading the README, we could definitely make things clearer.

spirv-builder depends on rustc_codegen_spirv crate directly

Yes, which is confusing, because it's defined as optional and I don't actually know where it's used, if it all. We certainly use rustc_codegen_spirv_types, but that of course isn't used for compiling.

I think of spriv-std as the thing that rustc_codegen_spirv compiles. It's like Rust's std but for rust-gpu. So whilst spriv-std doesn't define rustc_codegen_spirv as a dependency, they are intimately linked. And we can think of cargo-gpu as simply cargo, whilst it closely follows rustc it is not itself rustc. So both cargo and cargo-gpu manage the compiler, they are not the compiler itself.

But there the analogies end, because cargo-gpu deviates from cargo by both taking the roll of cargo and rustup. And spirv-std differs from std in that it also has the roll of defining the "rustc" (namely rustc-codegen-spirv in our case) version.

cargo-gpu installs rustc-codegen-spirv versions. A single cargo-gpu binary can install multiple versions of rustc-codegen-spirv for multiple shader crates. Indeed it's possible that the cargo-gpu project becomes complete at some point and will work perfectly well for all new rustc-codegen-spirv updates without cargo-gpu ever needing to be updated itself.

But again I could still very well be misunderstanding you. So please correct me.

[nazar-pc]

I'm even more confused. spirv-std has nothing to do with Rust-GPU/rust-gpu#302, spirv-std didn't change in that PR.

My understanding is that spirv-builder is building software for SPIR-V target, for which it needs rustc_codegen_spirv. So it builds rustc_codegen_spirv and uses it when calling rustc or cargo or however that works in practice. In turn, spirv-builder is what cargo-gpu itself uses, hence this PR.

cargo-gpu doesn't depend on rustc_codegen_spirv directly, just it doesn't depend on spirv-std either.

[tombh]

spirv-std has nothing to do with Rust-GPU/rust-gpu#302

As I mentioned, spirv-std differs from the conventional Rust std analogy, because spirv-std is how cargo-gpu tracks the version of rustc-codegen-spirv it needs to install. In effect spirv-std is rustc-codegen-spirv.

My understanding is that spirv-builder is building software for SPIR-V target, for which it needs rustc_codegen_spirv. So it builds rustc_codegen_spirv and uses it when calling rustc or cargo or however that works in practice.

Exactly right.

cargo-gpu doesn't depend on rustc_codegen_spirv directly, just it doesn't depend on spirv-std either.

Also right.

I'm probably still misunderstanding you. Do you believe that each shader crate needs to install its own dedicated version of cargo-gpu? And if not, how could each shader crate define the version of rustc-codegen-spirv that it needs?

[nazar-pc]

I looked at cargo-gpu code a bit more closely and I do see that it is looking at spirv-std in crate's dependency. This is quite surprising to me, primarily because cargo-gpu, rust-gpu and spirv-std should kind of be in sync, or else it is possible that cargo-gpu will find codegen version that is actually incompatible.

Now I see what you're saying, looks like upgrading spirv-std is the only thing needed in my case. Thank you for being patient and explaining this to me!

[tombh]

This is quite surprising to me, primarily because cargo-gpu, rust-gpu and spirv-std should kind of be in sync

I think that's very fair, especially because most of us are used to seeing cargo, rustc and rust-std all update in sync when we do rustup update.

Thank you for being patient and explaining this to me!

Thank you for your patience too! I'm 100% certain you're not the only one to go through this thought process and the conversation in this issue will be useful to future users.

It also shows that we could tweak our docs too.

[Firestar99]

Let me give this another try:

There's two ways to use rust-gpu currently:

• The "direct way" by using spirv-builder directly from the rust-gpu repo
• The new "cargo-gpu way" with cargo-gpu providing you a special instance of spirv-builder

The direct way very much works like you expect: The version of spirv-builder you are using defines the version of the codegen backend you'll get. They are tightly linked with a direct dependency on each other. But that dependency also requires that your entire project to be compiled with the toolchain version the codegen backend needs. (I prefer not to call it old since it is very much in use within the rust-gpu repo.)

Cargo-gpu breaks up the direct dependency of spirv-builder to the codegen backend, which allows your project to stay on stable with only the shader compilation using the specific toolchain version. The key observation is that spirv-builder doesn't actually need anything from the codegen backend, it merely needs the codegen backend compiled as a shared library (.so or .dll). This dependency is just a tool to ensure the "codegen backend shared library" is build in your target folder. But as long as you can somehow supply spirv-builder with a path to the shared library, the required toolchain version and a rustc target specification json file, it'll work. (And we must ensure spirv-builder is backwards compatible.)

Effectively, cargo-gpu is just a cache of codegen backend "installations", compiled on-demand depending on the version of spirv-std used, that configures spirv-builder to use these cached shared libraries. (And also a cmdline utility, which may be confusing.)

For more detail, have a look here:

• refactor: use SpirvBuilder directly, just manage rustc_backend_spirv dylibs #69 (contains an outline of the install and build procedures)

• cargo-gpu/crates/cargo-gpu/src/install.rs

  Lines 12 to 57 in e574460

  	/// Represents a functional backend installation, whether it was cached or just installed.
  	#[derive(Clone, Debug)]
  	#[non_exhaustive]
  	pub struct InstalledBackend {
  	/// path to the `rustc_codegen_spirv` dylib
  	pub rustc_codegen_spirv_location: PathBuf,
  	/// toolchain channel name
  	pub toolchain_channel: String,
  	/// directory with target-specs json files
  	pub target_spec_dir: PathBuf,
  	}
  	impl InstalledBackend {
  	/// Creates a new `SpirvBuilder` configured to use this installed backend.
  	#[expect(
  	clippy::unreachable,
  	reason = "it's unreachable, no need to return a Result"
  	)]
  	#[expect(clippy::impl_trait_in_params, reason = "forwarding spirv-builder API")]
  	#[inline]
  	pub fn to_spirv_builder(
  	&self,
  	path_to_crate: impl AsRef<Path>,
  	target: impl Into<String>,
  	) -> SpirvBuilder {
  	let mut builder = SpirvBuilder::new(path_to_crate, target);
  	self.configure_spirv_builder(&mut builder)
  	.unwrap_or_else(|_| unreachable!("we set target before calling this function"));
  	builder
  	}
  	/// Configures the supplied [`SpirvBuilder`]. `SpirvBuilder.target` must be set and must not change after calling this function.
  	///
  	/// # Errors
  	/// if `SpirvBuilder.target` is not set
  	#[inline]
  	pub fn configure_spirv_builder(&self, builder: &mut SpirvBuilder) -> anyhow::Result<()> {
  	builder.rustc_codegen_spirv_location = Some(self.rustc_codegen_spirv_location.clone());
  	builder.toolchain_overwrite = Some(self.toolchain_channel.clone());
  	builder.path_to_target_spec = Some(self.target_spec_dir.join(format!(
  	"{}.json",
  	builder.target.as_ref().context("expect target to be set")?
  	)));
  	Ok(())
  	}
  	}

• https://github.com/Rust-GPU/rust-gpu/blob/f58374079a72ee1f73bf2a9bdaed435b5e44e20e/crates/spirv-builder/src/lib.rs#L416-L421
• https://github.com/Rust-GPU/rust-gpu/blob/f58374079a72ee1f73bf2a9bdaed435b5e44e20e/crates/spirv-builder/src/lib.rs#L822-L829 (the comment is only referring to the "direct way")

(we should copy this into the docs)