Mesh Shaders Tracking Issue

[SupaMaggie70Incorporated]

Replaces #3018.

Spec

Progress

Current open PR(s) and other work

• Initial naga changes for mesh shaders #7930 - this is a behemoth PR that won't actually be merged. Smaller PRs will break off chunks of this and merge them individually
• Add mesh shading info to naga IR #8104 - adds naga IR and validation, broken off of Initial naga changes for mesh shaders #7930
• Update shader bencher to share some logic with snapshots #8108 - updates bencher behavior so that CI will be able to pass on my future PRs
• [hal/dx12] Mesh Shaders #8110 - DX12 support
• Add more complete MSL passthrough #8140 - better MSL passthrough for
• [hal/metal] Mesh Shaders #8139 - Metal support

naga

• Add mesh, task shaders to naga shader types so wgpu-hal can stop pretending they are compute shaders - Add mesh shader stages to wgt::ShaderStages and naga::ShaderStage #7292
• Support in WGSL frontend - Initial naga changes for mesh shaders #7930
• Validation - Initial naga changes for mesh shaders #7930 (partial, doesn't check against limits though)
• Support in SPIRV writer - Initial naga changes for mesh shaders #7930
• Support in HLSL writer
• Support in MSL writer
• Support in WGSL regurgitator
• Support in SPIR-V frontend
• Reflection info for pipeline validation

wgpu-hal backends

• Vulkan - Mesh shaders - initial wgpu hal changes #7089
• DX12 - Mesh Shading for DirectX 12 #7219 ([hal/dx12] Mesh Shaders #8110)
• Metal - [hal/metal] Mesh Shaders #8139 (incomplete)

Other

• Formal spec - Added mesh shading spec #7885
• wgpu API - Add mesh shading api to wgpu & wgpu-core #7345
• wgpu-core pipeline validation - Add mesh shading api to wgpu & wgpu-core #7345 (?)
• Limits & validation of mesh and task shader interface in wgpu-core #8003
• General tests- Add mesh shading api to wgpu & wgpu-core #7345
• More tests - line rendering, limits, etc
• Examples - Add mesh shading api to wgpu & wgpu-core #7345

Features

• Multiview mesh shaders #7262
• Per-primitive fragment shader inputs - Initial naga changes for mesh shaders #7930
• Primitive ID builtin (I haven't looked at or thought about this yet at all)
• Queries
• Point primitives(at least in vulkan)
• Finalize limits - Limits & validation of mesh and task shader interface in wgpu-core #8003

Current Priorities

• Multiview in wgpu-hal - very simple - Add multiview mesh shaders to wgpu-hal #7278
• Naga types - needed for wgpu-hal completeness(current code pretends they are compute shaders) - Add mesh shader stages to wgt::ShaderStages and naga::ShaderStage #7292
• Formal spec created and added to trunk - Added mesh shading spec #7885
• Experimental wgpu API - blocking many things - Add mesh shading api to wgpu & wgpu-core #7345
• Naga WGSL frontend - needed for proper wgpu implementation - Initial naga changes for mesh shaders #7930
• Naga SPIRV backend - needed for proper wgpu implementation - Initial naga changes for mesh shaders #7930
• DX12 in wgpu-hal - desired - [hal/dx12] Mesh Shaders #8110
• Naga reflection info - needed for proper wgpu implementation
• Lots of testing for wgpu - partly in Add mesh shading api to wgpu & wgpu-core #7345

Current workarounds

Possibly incorrect validation layers error

See KhronosGroup/Vulkan-ValidationLayers#10404.

Once everything is implemented correctly this shouldn't be an actual issue. All the built-ins (except the indices stuff) will need to be in the same struct anyway, since you can only have one such struct for builtins per 15.1.1.

Issues to work out

• How to implement multiview, what features, etc
• What information needs to be exposed through adapter limits
• How should point primitives be handled(probably not possible on directX?)
• Per primitive things on fragment shader

API differences

Task payload handling

• GLSL - only a single variable with the taskPayloadSharedEXT storage class per compilation unit(or none). This is then automatically passed to the entry point.
• SPIR-V - Variables can have the TaskPayloadWorkgroupEXT storage class. This is then optionally passed as an argument to OpEmitMeshTasksEXT. For mesh shaders the payload variable is inferred.
• HLSL - For mesh shaders, input is taken as in payload MeshPayloadStruct MeshPayload in function signature (only optional in pipelines without task shader). groupshared payload_t MeshPayload is taken as an argument to DispatchMesh in task shaders and is nonoptional. Additionally, struct must not be zero-sized in the task shader (a single bool is enough here in my testing). Therefore, the mesh shader must be different depending on whether there is a task shader, regardless of whether there is a task payload.
• MSL - Task shaders write the payload using object_data Payload& outPayload [[payload]] in the function parameters. Mesh shaders read them with object_data const Payload& payload [[payload]].

Current proposed method: in WGSL, an @payload(global_var) attribute for entry points. Potential drawbacks:

• May not be able to properly parse all SPIR-V shaders. This is a tradeoff, as HLSL/SPIRV have the potential payload output specified in the emitMeshTasks function, while MSL/GLSL has it specified in the entry signature. This should be fine anyway, but may require smarter parsing for SPIRV.

Limits

I'm unfamiliar with DX12, but it seems to be much more poorly documented in terms of what is/isn't allowed and what devices support what.

• It seems that DirectX 12 always limits you to no more than 128 threads in a mesh shader threadgroup. It is unclear what the limit for task shaders is, will probably need some testing
• DirectX 12 multiview is thankfully not implemented in WGPU
• Unclear to me what the max number of layers is?
• Pretty much everything is unclear to me

[SupaMaggie70Incorporated]

@cwfitzgerald
I’ve been thinking, we probably ought to have some way to indicate the “payload” for task and mesh shaders. I’m updating the original comment to add an attribute, @payload(some global variable). This would also probably mean we don’t need a new storage class. Let me know if you think there is a better option.

[cwfitzgerald]

Sounds fine - one thing to keep in mind that we want the IR to be compatible enough with how SPIR-V does stuff, such that spirv-in can eventually support mesh shaders. I don't know specifics about how mesh shaders work in spirv, but we should be careful to not require things that are too hard to implement in that frrontend. We' already require heroics in the spirv frontend for atomics.

[SupaMaggie70Incorporated]

@cwfitzgerald Thanks for the response. I'll try to keep a list of things that might be hard to implement in certain environments.