Create a performance guide

[gnzlbg]

PR #130 adds an skeleton for a performance guide about how to write efficient code with the library.

This issue is for discussing anything related to it.

[GabrielMajeri]

Since this is a book about performance and portability with SIMD, I don't like the idea of adding too much documentation, which would be a better fit for the standard library's docs.

• The std::arch docs and rustc book already mention RUSTFLAGS and the codegen options. I'd rather expand the documentation there than write too much about it in this book.

• If the user wants to learn how to do run-time feature detection, they should be linked to is_*_feature_detected docs. We shouldn't write its documentation, just use it in the sample code as a practical example.

• I also believe in a more Rust-by-example style. We could extensively comment and document the existing examples, and use them as case studies. It's easy to read about something in the book and forget about it; but if the users sees some actual code they can play with, and sees some benchmark results, they'll get a better grasp of what's going on.

Tl; dr: I'd like to keep performance-related tips and samples in this book, but try to keep general info related to using SIMD in Rust (which don't apply specifically to this crate) in the official documentation.

[gnzlbg]

I also believe in a more Rust-by-example style.

I think that style is fine for some parts of the guide. As a reference, ISPC has a performance guide that follows this style in some places, but diverges into a different style in others.

https://ispc.github.io/perfguide.html

The parts that show how to use the different features to write performant code can follow a Rust-by-example style.

---

I'd personally like the performance guide to contain most of what one needs to know to write "resonably performant" code using this library. Newcomers don't know everything that's currently available, and its hard for them to know whats there when the only way to discover is having to read the rustc book, the core docs, the std docs, etc. Particularly, when one advantage of packed_simd is that one doesn't necessarily need to use core::arch.

So I would prefer the perf guide to be "comprehensive" in that it mentions everything one needs to know. For things that are documented somewhere else, it can be as brief as possible, cross-linking those places, but still mentioning the most important bits.

Once we more or less have "everything we want to say" already written in some form, refactoring that across documentation sources and removing things from here is not that much extra work.

I wrote many of the stdsimd docs, but while there are some pieces that are equal, there are also others that differ subtly. So I think it would be easier to consolidate things after the fact.

[gnzlbg]

@GabrielMajeri do you want to be a co-maintainer of this crate ? (are you on IRC?) That basically involves helping with PR reviews, taking care of things when they break due to changes to rustc, CI, etc. I try to keep everything green, but an extra pair of hands would be great and it seems that this topic interests you and your recent contributions have been great.

[GabrielMajeri]

I would like to discuss some more about the flow / organisation of this book.

1. Try to switch to something like mdBook, and try setting up publishing to rust-lang-nursery.github.io

2. I've an idea of what to write in an introduction to people who haven't heard (a lot) about SIMD before but want to write performant Rust programs. The idea is to explain why SIMD is faster, and help determine when it isn't faster.

3. The current vertical vs horizontal operations could be greatly expanded by adding a section on "how to verticalize some algorithms". Examples such as

• the existing one on how to reduce (e.g. sum) a vector,
• how to break triangles in the Xs, Ys and Zs components and keep them in separate SIMD registers (e.g. aobench),
• how to use masks for conditions in SIMD loops (e.g. mandelbrot).

do you want to be a co-maintainer of this crate

I like contributing to this crate, and I still have some improvements and fixes planning to submit, but I'm not exactly a SIMD expert :) In fact, the whole reason for me contributing to this is that I am (fairly) new to SIMD and wanted to improve my understanding while contributing to open source (so thx for that).

[gnzlbg]

Try to switch to something like mdBook, and try setting up publishing to rust-lang-nursery.github.io

Good idea. This is now merged and up :)

I've an idea of what to write in an introduction to people who haven't heard (a lot) about SIMD before but want to write performant Rust programs. The idea is to explain why SIMD is faster, and help determine when it isn't faster.

The RFC also has an introduction to this, maybe we can reuse some stuff from there. I'd rather focus on this section at the very end though. Let's assume for now that whoever is trying to use this library has determined that its problem is suited for SIMD and just wants to use this library "properly". That is, we can add something here, but I'd prefer to keep it brief and focus our effort in the other parts for which documentation is more scarce.

The current vertical vs horizontal operations could be greatly expanded by adding a section on "how to verticalize some algorithms". Examples such as

That's a great idea. Currently the examples we have are just code, but because we also use them for benchmarks, it can be pretty steep to dive in, if one wants to use them to learn the library. Maybe we could have some sections about the code explaining how the different algorithms in the examples were "verticalized".

---

I think we should also add a section to the perf-guide to at least point users to the tools and methodology that can help them know if "they have done things right". That is, basic roofline analysis for their particular CPU (e.g. how to do the calculation for the sum_slice kernel or dot_product), how to obtain a rough performance ball park for their algorithm. How to obtain the assembly or llvm-ir of their algorithm (e.g. cargo asm, cargo llvm-ir, godbolt, playground, etc.), how to estimate the performance of the assembly for a particular architecture (e.g. IACA, LLVM MCA, etc.). How to fill in a performance bug ? (e.g. which code they wrote, which assembly they would expect, which assembly and llvm-ir they got, etc.).