Experimental std::simd

[fbusato]

Motivations

Modern GPU architectures are increasingly exposing fine-grained, single-thread SIMD capabilities to maximize throughput within individual CUDA threads. While GPU programming model strongly focuses on the SIMT model, newer hardware relies on specialized SIMD operations to saturate execution units. Some examples include:

• int16_t SIMD instructions DPX.
• FADDx2, FMULx2, FMAx2.
• Bfloat16x2 and Halfx2 intrinsics.
• 3-Input floating-point Minimum / Maximum.
• Integer Add X3 (IADD3).
• Integer dot product __dp4a.
• SIMD Video Instructions, e.g. vabsdiff4.
• SIMD within a register (SWAR) for integer types

C++26 std::simd provides a standardized abstraction to write vectorized code. This is a great opportunity to unify customized code to handle all variants and reduce CUDA software fragmentation. By adopting std::simd-like API, developers can write a single vectorized kernel that compiles to the optimal instructions for any GPU architecture.

PR Goals and Non-Goals

The PR aims to provide a basic implementation of std::simd and provide the foundation for future optimizations and extensions.

Advanced math and bit operations, e.g. std::abs , std::pow, std::popcount etc. , as well as std::complex binding, are outside the scope of the first PR.

Non-Goals:

• Fully-implement std::simd.
• Implement custom ABIs to target host vector instructions.

Implementation Notes

The implementation is based on the LLVM code experimental/__simd and extended to support the related C++ proposals:

• std::simd — merge data -parallel types from the Parallelism TS 2.
• simd issues: explicit, unsequenced, identity-element position, and members of disabled simd
• Exploration of namespaces for std::simd

Some optimizations are already exploited in the CCCL code, for example thread_simd.h and thread_reduce.h. They will gradually added to the implementation.

Partially address #30

[miscco]

This is not using SIMD on the host, is there any reason for that?

[fbusato]

because this is the first PR. Secondly, because we care more about GPU than CPU. Third, the feature is also experimental for other std libraries.

[github-actions]

🥳 CI Workflow Results

🟩 Finished in 15m 27s: Pass: 100%/42 | Total: 2h 47m | Max: 14m 49s | Hits: 99%/20431

See results here.