> the rust-gpu project has to emulate pointers[8], which we consider a blocking issue for most
HPC benchmarks.
Why is it a blocking issue?
I feel like this is very aligned with the goals of rust-gpu.
minraws•Aug 17, 2026
Pointers are sort of needed for high performance memory management for HPC targets for existing design patterns, maybe we can think of better solutions down the line but it's hard for me to say anything I just use/abuse CUDA pointers as well.
adgjlsfhk1•Aug 17, 2026
Julia has pretty good design heritage for how to deal with this sort of thing. you build the right abstractions and everything works (the main key is making sure the compiler elides bounds checks)
rfgplk•Aug 17, 2026
Fascinating how many people still overcomplicate offloading to GPUs.
MBCook•Aug 17, 2026
How so? I don’t know anything about this area.
konradha•Aug 17, 2026
What's the easy way here? Linking CUDA into your Rust binary?
I just read this tool up and down but still can't conceive of a use case.
foltik•Aug 18, 2026
No, it’s not relevant to simple GPU offload, stop spam linking your slop blog.
maxchisto•Aug 17, 2026
does anyone know Mojo well enough to comment how Rust + gpu-offload compares to it?
giancarlostoro•Aug 17, 2026
Mojo is not fully open sourced yet, but it will eventually be, would be an interesting comparison though.
maxchisto•Aug 17, 2026
Mojo's OSS status doesn't prevent us from evaluating its memory model, writing and benchmarking kernels in it, etc
giancarlostoro•Aug 17, 2026
Sure, and I realized after I posted the std lib is opened up, not sure how much of it will reveal the underlying Mojo specifics though.
Alephinitesimal•Aug 17, 2026
The NVIDIA+AMD support is the part I find really interesting. I know OpenMP and SYCL can already target multiple GPU vendors, but doing this while keeping Rust's safety model seems pretty compelling. I'm curious how portable the performance is in practice.
boywitharupee•Aug 17, 2026
is this mainly about making host binaries self-contained for heterogenous workloads?
also, seems like this is mostly targeted towards HPC audience?
whateverboat•Aug 17, 2026
> This module is under active development. Once upstream, it should allow Rust developers to run Rust code on GPUs. We aim to develop a rusty GPU programming interface, which is safe, convenient and sufficiently fast by default. This includes automatic data movement to and from the GPU, in a efficient way. We will (later) also offer more advanced, possibly unsafe, interfaces which allow a higher degree of control.
I really appreciate the work and the effort that went into this. However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?
aw1621107•Aug 17, 2026
> However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?
I think that will depend on the exact reason(s) C++ with LLVM offload didn't work out? If Rust differs from C++ in a way that addresses pain points/failure modes/etc. from the C++ attempt, for instance, then perhaps it isn't unreasonable to think Rust could succeed where C++ didn't (c.f., Mozilla's pre-Rust attempts to parallelize Firefox's CSS styling engine). Inversely, if Rust doesn't do things differently in the right way perhaps one might expect the effort to also not work out. Or maybe the problems are entirely non-technical and things could work out in either language.
erupti•Aug 18, 2026
> I think that will depend on the exact reason(s) C++ with LLVM offload didn't work out?
Are you making a claim or asking a question? ESL?
ux266478•Aug 17, 2026
> However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?
They're very different languages, with different semantics. Without reading more than the synopsis of the paper, they're 100% leveraging the substructural type system and will have a really tight requirement for you to use a certain kind of Rust code at the CPU/GPU boundary.
erupti•Aug 18, 2026
Unfortunately, the Rust description itself is inconsistent. It claims to be "sufficiently fast by default", yet "sufficiently fast" depends entirely on the requirements of a specific user project. And then it also plans to provide options that do not guarantee memory safety when the default speed is insufficient. It is already common for Rust projects to sprinkle memory unsafe code around when performance is needed.
aw1621107•Aug 18, 2026
> It claims to be "sufficiently fast by default", yet "sufficiently fast" depends entirely on the requirements of a specific user project.
I think that's why the "by default" is there; the goal is to offer a safe/convenient API that performs well enough that by default you don't need to reach beyond said safe/convenient API. And if you happen to be in a situation where the default performance of the safe/convenient APIs is insufficient, more advanced APIs will be provided.
It's a mirror of Rust's general design goals, if anything.
eptcyka•Aug 18, 2026
I wonder if they're looking to achieve easy speedups. I can see lots of value enabling performance gains where normally people wouldn't bother because it's too much effort. I don't think this will take away work from those who hand-optimise their kernels and scheduling, this is to enable GPU acceleration for those, who otherwise wouldn't.
boxed•Aug 17, 2026
Seems to work out well for Mojo, so I'd guess it's more an issue with C++.
jcelerier•Aug 17, 2026
> However, such an approach has previously not really worked for C++ with LLVM offload
... isn't Metal shading language just C++17 compiled with LLVM ? working on every Mac and iPhone in the world is not what I would call "not really worked". Likewise, SYCL works just fine.
mathisfun123•Aug 17, 2026
bruh lol this is so wrong and so confident i don't want to even attempt to explain how wrong you are.
> just C++17 compiled with LLVM
i invite you to attempt to compile/run absolutely any C++17 codebase on your iphone's GPU lol!
nish__•Aug 18, 2026
Wouldn't any metal app have to be written in C++17?
jcelerier•Aug 18, 2026
I mean "any C++ codebase" doesn't make sense in general. I run C++23 code on ESP32, that doesn't mean I'm gonna build KDE or chrome for it, and that doesn't make it any less C++.
pjmlp•Aug 18, 2026
For starters Metal Shading Language is based on C++14, so naturally any C++17 will fail.
pjmlp•Aug 18, 2026
It is C++14.
YuechenLi•Aug 17, 2026
So... why go through LLVM at all instead of having the MIR target PTX/HIP C directly then?
If they really wanted a vendor neutral solution for Rust GPU, that already exists: you write the CPU side code, including buffering, allocation, concurrency, etc through Vulkan binding and consume the compute kernel in SPIR-V from HLSL/GLSL/WGSL etc. As it stands, the way they use Rust here feels more like using it like TypeScript types/interfaces than anything else.
Again, the size of most operations that should be done on the GPU is known ahead of time before compilation, so it's very much possible to statically allocate memory at compile time instead of going through all this trouble to write what's essentially a Rust shaped DSL for GPU compute.
sanxiyn•Aug 17, 2026
People go through trouble to write Python-shaped DSL for GPU compute. We will go "why o why?", but apparently such things are necessary to succeed in the market.
YuechenLi•Aug 17, 2026
Well, I suppose fake Python is better than fake C++ at least.
But yeah, I think Python's dominance in science/ML will eventually pass, just as FORTRAN and Matlab did before.
minraws•Aug 18, 2026
Because it's convenient? Shader and vulkan semantics can be quite limiting and annoying to write.
Maybe it doesn't matter in a post AI world but perhaps it will allow better abstractions.
No need to yuck someone else's yum.
gmueckl•Aug 18, 2026
The compute shader side of Vulkan is actually fairly smooth compared to graphics. It's not really that different from low level CUDA or OpenCL. The Vulkan complexities overwhelmingly concern rasterization and raytracing.
bicepjai•Aug 17, 2026
I write all my code in Rust because I am a Rustacean. In many of my custom LLM inference engine projects, the biggest fight has always been bindings. I don’t want to maintain and write bindings; also, if I use an existing project that provides bindings, then I have to wait for the owner to update or fork it and then maintain it on top. It has been a big headache. Running Rust core on GPU sounds like something I will try from day one. Kudos to the team and will watch it closely.
Driftbench•Aug 18, 2026
Ownership tracking should map well to GPU memory lifetimes. That's one place Rust has a real edge over C++.
10 Comments
https://rustc-dev-guide.rust-lang.org/offload/internals.html https://github.com/rust-lang/rust/issues/131513
Why is it a blocking issue?
I feel like this is very aligned with the goals of rust-gpu.
This might be an relevant read: https://smolmachines.com/engineering/gpu-over-vsock
also, seems like this is mostly targeted towards HPC audience?
I really appreciate the work and the effort that went into this. However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?
I think that will depend on the exact reason(s) C++ with LLVM offload didn't work out? If Rust differs from C++ in a way that addresses pain points/failure modes/etc. from the C++ attempt, for instance, then perhaps it isn't unreasonable to think Rust could succeed where C++ didn't (c.f., Mozilla's pre-Rust attempts to parallelize Firefox's CSS styling engine). Inversely, if Rust doesn't do things differently in the right way perhaps one might expect the effort to also not work out. Or maybe the problems are entirely non-technical and things could work out in either language.
Are you making a claim or asking a question? ESL?
They're very different languages, with different semantics. Without reading more than the synopsis of the paper, they're 100% leveraging the substructural type system and will have a really tight requirement for you to use a certain kind of Rust code at the CPU/GPU boundary.
I think that's why the "by default" is there; the goal is to offer a safe/convenient API that performs well enough that by default you don't need to reach beyond said safe/convenient API. And if you happen to be in a situation where the default performance of the safe/convenient APIs is insufficient, more advanced APIs will be provided.
It's a mirror of Rust's general design goals, if anything.
... isn't Metal shading language just C++17 compiled with LLVM ? working on every Mac and iPhone in the world is not what I would call "not really worked". Likewise, SYCL works just fine.
> just C++17 compiled with LLVM
i invite you to attempt to compile/run absolutely any C++17 codebase on your iphone's GPU lol!
If they really wanted a vendor neutral solution for Rust GPU, that already exists: you write the CPU side code, including buffering, allocation, concurrency, etc through Vulkan binding and consume the compute kernel in SPIR-V from HLSL/GLSL/WGSL etc. As it stands, the way they use Rust here feels more like using it like TypeScript types/interfaces than anything else.
Again, the size of most operations that should be done on the GPU is known ahead of time before compilation, so it's very much possible to statically allocate memory at compile time instead of going through all this trouble to write what's essentially a Rust shaped DSL for GPU compute.
But yeah, I think Python's dominance in science/ML will eventually pass, just as FORTRAN and Matlab did before.
Maybe it doesn't matter in a post AI world but perhaps it will allow better abstractions.
No need to yuck someone else's yum.