ReSTIR FG: Real-Time Reservoir Resampled Photon Final Gathering

Introduction

This repository contains the source code and an interactive demo for the following EGSR paper:

ReSTIR FG: Real-Time Reservoir Resampled Photon Final Gathering
René Kern, Felix Brüll, Thorsten Grosch
TU Clausthal

This prototype implements ReSTIR FG, an efficient real-time global illumination algorithm that combines photon final gaterhing with reservoir resampling (ReSTIR). Additionally, our ReSTIR FG is able to display caustics in real-time. For direct light, RTXDI is used. This prototype also contains a denoised prototype using DLSS and NRD.

This repository also includes a simplified version of ReSTIR FG, called ReSTIR FG Lite. This variant is designed to make it easier to learn the core algorithm and follows the notation used in the "A gentle Introduction to ReSTIR" course. For more details, see here.

This project was implemented using NVIDIA's Falcor rendering framework. See README_Falcor.md for the readme provided with Falcor.

You can download the executable demo from the Releases Page, or build the project by following the instructions in Building Falcor or the build instructions in the original readme.

Teaser:

Contents:

• ReSTIR FG Lite
• Demo usage
• Testing with more Scenes
• Falcor Prerequisites
• Building Falcor

ReSTIR FG Lite

ReSTIR FG Lite is a stripped-down version of our base variant, designed to highlight the core algorithm in a simpler form. The implementation follows the notation used in the "A gentle Introduction to ReSTIR" course.

The algorithm works as follows:

1. VBuffer Pass - Acquire the first surface hit.
2. Trace Photons (Shader)
   • Photons are only stored on diffuse surfaces. Photons are stored only on diffuse surfaces (classification is based on a roughness threshold, default = 0.25).
   • Photons are classified as Global (last hit diffuse) or Caustic (last hit specular).
   • Global photons are stochastically rejected with a fixed probability to reduce acceleration structure build time and relatively increase caustic photon density.
3. Build the Photon Acceleration Structure
   • Photons are stored as Axis-Aligned Bounding Boxes (AABBs) with their collection radius as extent.
4. Generate Initial Final Gather and Caustic Reservoirs (Shader)
   • Reservoirs are initialized at the first diffuse surface. (For Final Gather, an additional ray is traced)
   • If the VBuffer surface is not diffuse, a ray is traced until one is found. (Or maximal ray depth is reached)
   • If the first surface is non-diffuse, caustic reservoirs also include Global photons with path length 0 (direct illumination), since RTXDI cannot properly handle these samples.
   • The RTXDI surface is also set in this step.
5. Direct Light Sampling and Resampling - performed with RTXDI.
6. Resample Final Gather Reservoirs with spatiotemporal samples (Shader).
7. Resample Caustic Reservoirs with (spatio)temporal samples (Shader).
8. Evaluate All Reservoirs (Shader).

(Shared structs and utility functions are located here).

Demo usage

After downloading the demo from the release page, execute StartDemo.bat and select the desired algorithm and scene. We provide four scenes with the Demo, two are included in the git repo in the Models folder (VeachAjar and Sibernik). The other two scenes need to be downloaded separately (Kitchen and Bistro) from the Releases Page and unziped into the Models folder. For more scenes, see the Testing with more Scenes section.

To change the settings of our algorithm, navigate to the ReSTIR_FG group in the UI. In addition to ReSTIR FG, we have implemented Final Gather and ReSTIR GI, which can be switched in the UI. For more information about a setting, hover over the (?).

Controls:

• WASD - Camera movement
• Left Click + Mouse movement - Change camera direction
• Shift - Speed up camera movement
• Q, E - Camera Down / UP
• P - Opens the profiler that shows the Rendertime for each Pass ('ShadowPass' is ours).
• F9 - Opens the time menu. Animation and camera path speed can be changed here (Scale).
• F6 - Toggels Graphs UI menu (Enabled by default)

Note: We use the Lambertian diffuse BRDF to enable direct comparison with the ReSTIR PT and Suffix ReSTIR prototypes. To switch to Falcors's default diffuse BRDF, disable ReSTIR_FG -> Material Options -> Use Lambertian Diffuse BRDF.

Testing with more Scenes

Testing with other scenes is possible. The following points should be noted when loading other scenes:

• Load the ReSTIR_FG renderscript:
  • Using the .bat file (ReSTIRFGDemo_NoScene.bat)
  • Directly in Mogwai with File->Load Script. The render pass scripts are in the scripts folder (Data/scripts in demo).
• A scene can be loaded in Mogwai with File->Load Scene.
• ReSTIR FG supports emissive materials and analytic point/spot lights. Photons are not distributed from environment maps or directional lights.
• We automatically set the radius depending on the scene extent. This is not optimal and may require manual adjustments. Too large radii can cause performance issues, while too small radii can cause correlations.

Falcor supports a variety of scene types:

• Falcor's .pyscene format (more details)
  • e.g. NVIDIA ORCA
• GLTF
  • Brightness of emissive materials may need to be adjusted.
• FBX
  • Often need manual adjustments for emissive and glass materials.
• Many PBRT V4 files:
  • e.g. Benedikt Bitterli's Rendering Resources or PBRTv4 scenes repo
  • May require manual adjustments of materials, as not all materials match Falcor's material model.

Falcor Prerequisites

• Windows 10 version 20H2 (October 2020 Update) or newer, OS build revision .789 or newer
• Visual Studio 2022
• Windows 10 SDK (10.0.19041.0) for Windows 10, version 2004
• A GPU which supports DirectX Raytracing, such as the NVIDIA Titan V or GeForce RTX
• NVIDIA driver 466.11 or newer

Optional:

• Windows 10 Graphics Tools. To run DirectX 12 applications with the debug layer enabled, you must install this. There are two ways to install it:
  • Click the Windows button and type Optional Features, in the window that opens click Add a feature and select Graphics Tools.
  • Download an offline package from here. Choose a ZIP file that matches the OS version you are using (not the SDK version used for building Falcor). The ZIP includes a document which explains how to install the graphics tools.
• NVAPI, CUDA, OptiX (see below)

Building Falcor

Falcor uses the CMake build system. Additional information on how to use Falcor with CMake is available in the CMake development documetation page.

Visual Studio

If you are working with Visual Studio 2022, you can setup a native Visual Studio solution by running setup_vs2022.bat after cloning this repository. The solution files are written to build/windows-vs2022 and the binary output is located in build/windows-vs2022/bin.