Tracking issue for high-level compiler overview rework

[jieyouxu]

This is a tracking issue for high-level compiler overview chapter rework. This is not a serious r-l/r kind of tracking issue, so discussions/feedback/suggestions totally welcomed (opening issues for specific topics also great)!

TODO: I plan to edit this issue soon, opening now so I don't forgor.

Rationale

TODO

Implementation steps

TODO: fill out this section a bit more

• Identify the intended audience.
  • Ideally, it should be useful to both new contributors and veteran contributors alike! Even veteran contributors will have areas they don't know well about, and this should equally cater to them.
• Determine a suitable overall structure for the overview. How do we introduce rustc to an interested contributor who never read rustc's source to help them aboard?
  • Current idea: I plan to describe rustc as a "cute little command-line tool that gradually lowers from one intermediate representation to another, but with a starting input of 'source code' and output of 'machine code'"¹
  • How does this compare to the previous idea of information flow between crates sub-systems? It's more faithful to what actually happens and is compatible with query system being not coarse-grained IR-to-IR-in-fullness (i.e. IRs can be gradually lowered, in parts, on demand) and is also compatible with incr comp. It's also less of a spaghetti of arrows if I want to draw some arrows between "nodes" in what I want to describe.
• Figure out how to talk about cross-cutting subsystems (diagnostics, type checking, traits, lints, error handling, query system)
• Figure out if we should also have a listing that briefly introduces the intended purpose of each compiler crate ². For each compiler crate, produce a brief TL;DR:
  • How does it support the lowering and various analyses? What purposes does it serve?
  • How does it fit into the "big picture"?
  • Main entry points? (Link to source code, pinned to specific commit, add a date-check annotation)
• Actually write the thing.

Related issues

• Overview Chapter Followup #674
• Improve compiler top-level description in rustc-dev guide #1691

Implementation history

• [Draft] Rework the "Overview of the Compiler" chapter #1955
  • This was the previous approach where I tried to describe the compiler as subsystems and their information dependency, but I quickly discovered that fell apart when we tried to factor in e.g. the trait system or type check or the query system.
• Add overview.drawio.svg #1725 (overview diagram, we can probably absorb it / message it as suitable)

Footnotes

1. obviously it's more complicated than that, but I personally find significant value in "xxx oversimplified" because having a rough high-level picture makes it easy for you to know what you're getting into, and then to become an expert by knowing all the "but acktually" cases ↩

2. I'm envisioning a distinction between overview/intro.md versus overview/crates/rustc_middle.md etc. etc. We can also go ask the experts for how they think about each crate (for a smol TL;DR) in (possibly) weekly/monthly dev-guide review sessions! ↩

[jieyouxu]

(Unfortunately I don't have the bandwidth to pursue this atm, feel free to take over)