-
Notifications
You must be signed in to change notification settings - Fork 63
Description
I've been looking at rust-gpu last two days and since toolchain upgrades are non-trivial and sometimes take a while to actually happen, I'm wondering if it would be possible to limit toolchain requirement to just the code that is compiled for GPU.
Specifically, SpirvBuilder
calls cargo or rustc internally and could easily specify something like +nightly-whatever
in CLI. This would result in backend matching what SPIR-V backend expects, while allowing the rest of user's workspace using whatever toolchain it uses by default.
Override with +
has the highest priority, so this should work nicely.
Otherwise it requires much more elaborate setup with separate SPIR-V compilation and usage steps, probably separate workspaces, etc.
UPD: Looks like there is already toolchain_overwrite
on the builder, now I'm wondering why its value doesn't default to this workspace's rust-toolchain.toml
🤔
UPD2: This is exactly what I'm talking about, it should just use correct toolchain by default and not check current toolchain/generate an error if there is a single version supported anyway:
Compiling rustc_codegen_spirv v0.9.0 (https://github.com/Rust-GPU/rust-gpu?rev=e6d017d5504c4441a84edcc27f4eca61de6fc8cf#e6d017d5)
error: failed to run custom build command for `rustc_codegen_spirv v0.9.0 (https://github.com/Rust-GPU/rust-gpu?rev=e6d017d5504c4441a84edcc27f4eca61de6fc8cf#e6d017d5)`
Caused by:
process didn't exit successfully: `/home/nazar-pc/.cache/cargo/target/debug/build/rustc_codegen_spirv-3c6c975d26047f08/build-script-build` (exit status: 1)
--- stdout
cargo:rerun-if-env-changed=RUSTGPU_SKIP_TOOLCHAIN_CHECK
--- stderr
error: wrong toolchain detected (found commit hash `c68340350c78eea402c4a85f8d9c1b7d3d607635`, expected `b19329a37cedf2027517ae22c87cf201f93d776e`).
Make sure your `rust-toolchain.toml` file contains the following:
-------------
[toolchain]
channel = "nightly-2024-11-22"
components = ["rust-src", "rustc-dev", "llvm-tools"]
-------------
Exit code 101
UPD3: This is even more problematic. Why does it check toolchain components if I explicitly used `` feature so it doesn't do that? I really don't want to add rustc-dev
to the workspace since I'll not be using it anyway 😕
error: failed to run custom build command for `rustc_codegen_spirv v0.9.0 (https://github.com/Rust-GPU/rust-gpu?rev=e6d017d5504c4441a84edcc27f4eca61de6fc8cf#e6d017d5)`
Caused by:
process didn't exit successfully: `/home/nazar-pc/.cache/cargo/target/debug/build/rustc_codegen_spirv-909afb669bc3840e/build-script-build` (exit status: 1)
--- stdout
cargo:rerun-if-env-changed=RUSTGPU_SKIP_TOOLCHAIN_CHECK
--- stderr
missing `rustc-dev` component from toolchain `nightly-2025-06-19-x86_64-unknown-linux-gnu` (at /home/nazar-pc/.rustup/toolchains/nightly-2025-06-19-x86_64-unknown-linux-gnu)
warning: build failed, waiting for other jobs to finish...
Exit code 101
UPD4: Looks like I hit #104 in the end.
Activity
LegNeato commentedon Jun 24, 2025
It is totally possible to do this and what most folks do! While it is currently manual, we have an alpha tool that automates it: https://github.com/Rust-GPU/cargo-gpu.
nazar-pc commentedon Jun 24, 2025
I'm not sure it does what I need here.
I'd like to build shaders in
build.rs
and I want to specify a toolchain to use in there (because toolchain of the workspace will certainly be different). Right now inclusion ofrustc_codegen_spirv
dependency triggers itsbuild.rs
, which expects a specific toolchain version. What I'd like here is to compilerustc_codegen_spirv
itself for a custom toolchain too.Probably something like
rustc_codegen_spirv_builder
that I can call programmatically and then setbuilder.rustc_codegen_spirv_location
and other stuff manually.LegNeato commentedon Jun 24, 2025
cargo gpu
's intention was to support the exact use-case you describe, but it is still alpha and may not be there. But the idea of "compile CPU code with a modern rust, GPU code with the specific nightly rust-gpu needs" is a supported workflow that most projects use. We haven't yet written it up, and cargo gpu is supposed to automate most if not all of it.@Firestar99 , you use
cargo-gpu
inbuild.rs
, right? It wasn't how it was intended to be used but I believe it works today. Can you give some insight here?nazar-pc commentedon Jun 24, 2025
Interesting! It would have been nice if it wasn't necessary to use a custom wrapper, especially since the project has a lot of crates and only a single one of them has GPU code. I also don't see if it could run things like
clippy
,nextest
and others that I use regularly.In the end I see the value for workspace that is primarily GPU-focused, but otherwise I'd strongly prefer for
cargo build
to be able to do the right thing. The more straightforward it is, the better. I'll experiment some more tomorrow and share if I come up with a satisfactory solution.Firestar99 commentedon Jun 25, 2025
See Rust-GPU/cargo-gpu#71 for sample code :D
Otherwise, you should be able to run clippy and nextest just fine, with the target being your CPU. Maybe this isn't obvious: you can have your CPU crate depend on the shader crate, which allows you to reuse all your non-GPU specific structs and algorithms (Limitations:
#[repr(C)]
and don't useVec3
). Similarly, you can test non-GPU specific code with normal test and run them on the CPU, only the entry points are disabled and GPU intrinsics don't work.nazar-pc commentedon Jun 25, 2025
I meant that by simply depending on
rustc_codegen_spirv
my crate was not easy to compile with something likecargo clippy --all-targets
in a workspace.Rust-GPU/cargo-gpu#71 seems to be exactly what I need, will give it a try, thanks!
Firestar99 commentedon Jun 25, 2025
Note that clippy currently fails in projects with such a build script... Rust-GPU/cargo-gpu#77
nazar-pc commentedon Jun 25, 2025
There has been some workarounds needed (including for Clippy), but in the end I came up with this
build.rs
using latestcargo-gpu
library revision:This crate is basically both shader and host conditionally compiled, and shader itself is embedded into the library at compile time:
Hopefully less boilerplate will be needed in the future.
Firestar99 commentedon Jun 25, 2025
Some notes on your scripts:
CARGO_TARGET_DIR
will cause the shaders to be build directly intarget
and not intarget/spirv-builder
rust-gpu/crates/spirv-builder/src/lib.rs
Lines 432 to 436 in e6d017d
&[u32]
properly included: https://docs.rs/wgpu/latest/wgpu/enum.ShaderSource.html#variant.SpirVnazar-pc commentedon Jun 25, 2025
Yes, in fact I tried to gate it for non-spirv target, but Cargo ignored that part, likely because build script is compiled for native platform (will try to add internal feature to exclude it).
True, but it is probably fine in my case, I'll make a split if it ever becomes a bottleneck.
Yes, but it is set in my
~/.profile
globally, not just for this project, so ideallycargo-gpu
would handle it more gracefully, hence Rust-GPU/cargo-gpu#90Yeah, this is why I'm setting
target_dir_path
for shader compilation, so it is isolated in its own small place.And I did reach out to them first, but neither function nor macro are
const
. My solution though will result in a desired data structure being embedded into the binary with compile-time checks. I was lazy to check for magic bytes (andwgpu
wasn't nice enough to make the constant public), but it is good enough.nazar-pc commentedon Jun 25, 2025
This this before calling building shader:
Together with this:
I expected it to not compile
cargo-gpu
, but it doesn't work: rust-lang/cargo#4423