Add FSR3 Upscaler support

[HydrogenC]

Overview

This PR only introduces FSR3 UPSCALER, which means FRAME GENERATION is irrelevant. In FSR 3.1, upscaler and frame gen is fully decoupled, so we can introduce either of them independently of the other.

This PR adds support for Fidelity Super Resolution (FSR) 3.1.4 on top of the common Fidelity FX interface introduced in #111512 .
The PR also contains improvements to and update to FSR1/2 and fundamental changes to the FSR interface, see description in #111512.

When the upscale multiplier is large (rendered/upscaled<0.7), you may expect glitches on faraway objects. Such glitches are also seem on the existing FSR2. This is a algorithm limitation of the FSR temporal upscaling, not likely a artifact caused by the implementation of this PR.

Improvement to FSR2

This PR adds a new option called "FSR auto generate reactive mask". If checked, FSR 2/3 will also run an additional reactive mask generation mask, which considerably reduces the flickering when dealing with transparent objects. The pass takes an opaque-only color attachment as input, so the color attachment is copied and saved between opaque render and transparent render. The additional pass could also be enabled for and benefit the existing FSR2 effect. The stabilization is quite conspicuous, for both FSR 2 and 3 (see comparison in comment below).

Tests

Tested on Windows Vulkan & DX12, other platforms untested.

Benchmarks

	FSR2 2x	FSR3 2x	FSR2 Native	FSR3 Native
GPU Time (ms)	0.27	0.33	0.41	0.47
Visuals

Todos

• Investigate and fix occasional crash - Solved
• Transparent objects flickers quite worse - Solved

[mrjustaguy]

At Native Quality, and FSR 2.3 is an upgrade over 2.2 on a few bright elements in motion in my Game (that with 2.2 tend to exhibit an unusual darkening until the motion stops), However 3.1 is... Bad...

Like this artifact from Flashing Lights happens a LOT in 3.1 but not in 2.3/2.2

though Mind you 2.3/2.2 tend to have leftover fireflies when something was very sharply bright/dark but this is also present in 3.1

I have been Unable to test if FSR 4 Works, as been unable to compile DX12 to run on my Brother's 16 gig RX 9060 XT, which for Native rendering is the only potential saving grace for RDNA4 (and possibly eventually RDNA3 and below, and even other vendors, depending on what kind of official support AMD has with their int8 version) users

Upscaling (at 0.5 render scale) differences are mainly in Transparency Handling which is better and less pixelated and also seems to preserve antialiasing a bit better in some scenarios, but the issue created in 3.1 at Native just absolutely murders any of the Quality Gains.

However I haven't been able to see a difference in Performance Between Native FSR 2.3 and 3.1, which I expected FSR 3.1 to take 50% longer according to AMD's own Performance Estimates (I'm getting 2.2ms on both at native res on an RX 6600)
Though these performance Claims can only be estimated Upscale to Upscale and not Native to Native as FSR 2 doesn't officially do Native and no such performance estimates thus exist, so I've used FSR 3.1 4k Quality on 6800xt vs FSR 2 4k Quality on 6800xt official numbers to get the expected Performance Delta
Trying out with quality Upscaling results in ~1.7ms on both FSR 3.1 and FSR 2.3 for me on the RX 6600 at 1440p

https://gpuopen.com/fidelityfx-super-resolution-3/
https://gpuopen.com/fidelityfx-superresolution-2/

All This leads me to believe something is Wrong with this FSR 3.1 Implementation. (though the Light issue with 3.1 I have observed to a lesser extent in Other FSR 3.1 games such as Everspace 2 and Star Wars Outlaws neither of which have FSR 2 for comparison, and may be an issue with it and not the implementation of it, that my Game is suffering particularly heavily from)

[HydrogenC]

@mrjustaguy As for the lighting artifact, I could also see some extent on my side. If possible, could you provide a test scene in which the problem you describes is clearly observable, so that I could investigate the reason that could have caused it?
As for the performance part. The metrics are collected within Godot editor, which features a small viewport (1548x768), so it isn't surprising that the time numbers are small. Before I submitted this PR, I have made frame captures within NVIDIA Nsight Graphics and I can prove that all passes are present. I checked the intermediate images and they looks normal. So if there's a problem, it must be an input problem, which is unrelated with the performance considerations that you stated.

[mrjustaguy]

Sure, Give me a little to make all the MRPs for all of the comparisons I'm making in an edit on this in a bit.

Do keep in mind however the Light issue described above is an FSR 3.1 issue given the other games also suffer from, just to a lesser degree and it may be just that content differences are making it act up far more in my testing.

One thing that may be an implementation issue is the Transparency-Opaque transition AA falling apart..

Also while writing this I started experiencing heavy Godot instability with FSR 3.1 enabled

Edit: OK, fun little observation. In one of the Opaque AA stress tests where FSR 3.1 performs really well compared to FSR 2.3 I've managed to break the AA completely (turns totally jittery and unstable) when adding a Material to one of the Planes (even though it's not in the background of the AA that goes bonkers) and shortly after that it crashes

[HydrogenC]

One thing that may be an implementation issue is the Transparency-Opaque transition AA falling apart..

This is highly possible. FSR relies on something called the Transparency & Composition mask. In the previous FSR2 implementation, this mask isn't provided, maybe because of rendering architectural reasons. So I still kept it unprovided within FSR3. It's possible that FSR3 is more reliant on this mask to produce good-looking effects.

shortly after that it crashes

Do you mean Godot crashing? Is there a log or something I could check for?

[mrjustaguy]

That's probably the reason, I have never seen this issue with FSR 3.1 games

Really the only two issues are that and the Lighting Lag/Semi Clear up issue (with both really being massive pain points in my Space Game, as Transparent Background and Lights Flashing in and out a ton hahaha)

Other testing that I have done since however hasn't shown any more problems and generally it is an improvement if you ignore the Transparencies (specifically due to the Opaque-Alpha Transition) and rapid Light Changes.. anyhow back to MRP-ing

I'll look for Logs

[HydrogenC]

I'll look for Logs

I have met an occasion crash case denoted as:

ERROR: FATAL: Index p_index = 32 is out of bounds (count = 32).
   at: LocalVector<class RID,unsigned int,0,0>::operator [] (D:\Code\godot\core/templates/local_vector.h:205)

If the one you met is the same, then I could confirm that it's not purely occasional but somehow reproducible and will seek to investigate it.

[mrjustaguy]

I have found a way to Reproduce it consistently, by adding a MeshInstance, setting mesh to Plane, and then duplicating the instance

[mrjustaguy]

This MRP Showcases the Light issue quite well, and the Alpha-Opaque Instability

EDIT: Here's the proper Test Scene, with the Alpha Opaque Instability, Forgot to save that last addition before shipping it into the zip... 🙄
FSR.zip

[HydrogenC]

This MRP Showcases the Light issue quite well, and the Alpha-Opaque Instability

EDIT: Here's the proper Test Scene, with the Alpha Opaque Instability, Forgot to save that last addition before shipping it into the zip... 🙄 FSR.zip

Thanks a lot! I would investigate if it's a implementation bug or just a artifact caused by lack of composition mask input. If it's the latter, we would have little way to improve it.

[mrjustaguy]

Some more FSR 3.1 testing VS FSR 2.3 and observations

Low Quality Soft Shadows don't leave Significantly less fireflies in 3.1, and now Soft Shadows on Low give results beating Soft Shadows Ultra at equivalent blur (when it Comes to Fireflies, the Soft Shadows do look a little grainy, but stable)

Another thing that I have to note is that Once you take out Alpha out of the equation, the AA is far more stable both when static and in motion, and I have failed to produce FSR 2's AA collapse that occurs in motion in some instances (not to be confused by #104622)

Also some instances of Ghosting that I have been consistently able to reproduce with FSR 2 have been eliminated (around Caused by Smooth Metalic Objects moving close to an Explosion that's lighting it up and the background Object)

I have been unable to find any further issues, so For most instances I'd go with 3.1 over 2.3, and I'd only Pick 2.3 in cases of Heavy Alpha transparency usage covering large areas of the screen constantly and Fast Moving/changing Lights, as it stands right now.

[HydrogenC]

I have been unable to find any further issues, so For most instances I'd go with 3.1 over 2.3, and I'd only Pick 2.3 in cases of Heavy Alpha transparency usage covering large areas of the screen constantly and Fast Moving

Nice to hear that. So FSR3 does have improvements over FSR2. The first thing I'm going to do is to investigate those crashes which are AFAIK caused by resource use after free. And once that is fixed, this PR is mergable. Then I will try if I could generate a transparancy & composition mask without overhauling the rendering pipeline, to relieve the alpha problem you encountered. And if it's possible, I'll also add that mask for FSR2.

[HydrogenC]

@mrjustaguy I discovered and fixed the bug that caused the instability, which is actually an upstream (#111512 ) problem. Now I fixed it upstream and rebased this PR. Could you please test again whether the stability improves and that Godot no longer crashes?
If there isn't a stability problem present now, then we can inform the devs that this PR is functionally complete and ready for review. Your previous tests has already proved that FSR3 in Godot indeed has improvements over the current FSR2.
As for the alpha problem, I will keep investigate it. I will open a new PR if I find ways to relieve it. As this problem is not solely applicable to FSR3, but also related to the existing FSR2 implementation, and it would require changes to the core pipeline (which is out of scope for this PR), and that such improvements would be unlikely to introduce API breaks, I would be prone to do that in a new PR.

[mrjustaguy]

I will be able to confirm it in a little over half an hour when it builds on my weak lil quad core

[jcostello]

I get this error while trying to build

thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1050:9: error: 'wcscpy_s' was not declared in this scope
 1050 |         wcscpy_s(clearJob.jobLabel, L"Clear Resource");
      |         ^~~~~~~~
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1078:9: error: 'wcscpy_s' was not declared in this scope
 1078 |         wcscpy_s(clearJob.jobLabel, L"Clear Reconstructed Previous Nearest Depth");
      |         ^~~~~~~~
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1086:17: error: 'wcscpy_s' was not declared in this scope
 1086 |                 wcscpy_s(clearJob.jobLabel, L"Clear Spd Atomic Count");
      |                 ^~~~~~~~
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1129:26: error: '_countof' was not declared in this scope
 1129 |         for(int i = 0; i<_countof(aliasableResources); ++i)
      |                          ^~~~~~~~
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1138:13: error: 'wcscpy_s' was not declared in this scope
 1138 |             wcscpy_s(clearJob.jobLabel, L"Clear Spd Atomic Count");
      |             ^~~~~~~~
In file included from thirdparty/amd-ffx/ffx_interface.h:25,
                 from thirdparty/amd-ffx/ffx_fsr3upscaler.h:26,
                 from thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:27:
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp: In function 'FfxErrorCode ffxFsr3UpscalerContextCreate(FfxFsr3UpscalerContext*, const FfxFsr3UpscalerContextDescription*)':
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1209:54: error: static assertion failed: sizeof(FfxFsr3UpscalerContext) >= sizeof(FfxFsr3UpscalerContext_Private)
 1209 |     FFX_STATIC_ASSERT(sizeof(FfxFsr3UpscalerContext) >= sizeof(FfxFsr3UpscalerContext_Private));
      |                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
thirdparty/amd-ffx/ffx_assert.h:147:52: note: in definition of macro 'FFX_STATIC_ASSERT'
  147 | #define FFX_STATIC_ASSERT(condition) static_assert(condition, #condition)
      |                                                    ^~~~~~~~~
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1209:54: note: the comparison reduces to '(524288 >= 837928)'
 1209 |     FFX_STATIC_ASSERT(sizeof(FfxFsr3UpscalerContext) >= sizeof(FfxFsr3UpscalerContext_Private));
      |                       ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
thirdparty/amd-ffx/ffx_assert.h:147:52: note: in definition of macro 'FFX_STATIC_ASSERT'
  147 | #define FFX_STATIC_ASSERT(condition) static_assert(condition, #condition)
      |                                                    ^~~~~~~~~
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp: In function 'FfxErrorCode ffxFsr3UpscalerContextGenerateReactiveMask(FfxFsr3UpscalerContext*, const FfxFsr3UpscalerGenerateReactiveDescription*)':
thirdparty/amd-ffx/ffx_fsr3upscaler.cpp:1390:5: error: 'wcscpy_s' was not declared in this scope
 1390 |     wcscpy_s(jobDescriptor.srvTextures[0].name, pipeline->srvTextureBindings[0].name);
      |     ^~~~~~~~
scons: *** [bin/obj/thirdparty/amd-ffx/ffx_frameinterpolation.linuxbsd.editor.dev.x86_64.o] Error 1
scons: *** [bin/obj/thirdparty/amd-ffx/ffx_fsr3upscaler.linuxbsd.editor.dev.x86_64.o] Error 1
./thirdparty/amd-ffx/ffx_types.h:887:27: warning: 's_FfxViewDescInit' defined but not used [-Wunused-variable]
  887 | static FfxViewDescription s_FfxViewDescInit = { false, FFX_RESOURCE_VIEW_DIMENSION_TEXTURE_2D, -1, -1, -1, L"" };
      |                           ^~~~~~~~~~~~~~~~~
./thirdparty/amd-ffx/ffx_types.h:887:27: warning: 's_FfxViewDescInit' defined but not used [-Wunused-variable]
  887 | static FfxViewDescription s_FfxViewDescInit = { false, FFX_RESOURCE_VIEW_DIMENSION_TEXTURE_2D, -1, -1, -1, L"" };
      |                           ^~~~~~~~~~~~~~~~~
./thirdparty/amd-ffx/ffx_types.h:887:27: warning: 's_FfxViewDescInit' defined but not used [-Wunused-variable]
  887 | static FfxViewDescription s_FfxViewDescInit = { false, FFX_RESOURCE_VIEW_DIMENSION_TEXTURE_2D, -1, -1, -1, L"" };
      |                           ^~~~~~~~~~~~~~~~~
Ranlib Library bin/obj/scene/libscene.linuxbsd.editor.dev.x86_64.a ...
./thirdparty/amd-ffx/ffx_types.h:887:27: warning: 's_FfxViewDescInit' defined but not used [-Wunused-variable]
  887 | static FfxViewDescription s_FfxViewDescInit = { false, FFX_RESOURCE_VIEW_DIMENSION_TEXTURE_2D, -1, -1, -1, L"" };
      |                           ^~~~~~~~~~~~~~~~~
scons: building terminated because of errors.

Ubuntu 25.4

[mrjustaguy]

The instability seems to have been resolved for me

[mrjustaguy]

Ha. I've figured out why the noisy light issue is happening.

FSR 3.1 is actually picking up on EXTREMELY Subtle Noise Typically caused by Shadow Acne, and that's just making the Algorithm go Crazy.
There is still an issue but of a different type in the MRP where the Light's Specular Lobe seems a little deformed and ghosty

(for reference the old result with lower Directional Light Shadow Bias causing Acne)

So technically for Both Problems that made it look awful I have been able to work around the issues while maintaining all the visual aspects of it, by making the Background Opaque and Taking Extra Steps to Reduce Input Noise

[HydrogenC]

I noticed that there are some updates in the last rendering meeting, mentioning converting upscaler to something like a compositor effect #80925. That would be a better architecture, as long as it exposes necessary attachments. When the refactor is done, I would be glad to revive this PR in the new integration style.

[coffeel0RD]

any new info about this?

[HydrogenC]

any new info about this?

As far as I know this PR won't be adopted in the current form according to the rendering meeting.
The upscaler system is currently subject to refactor (see #114670). Once that refactor is done, I will hopefully migrate this PR to the new custom upscaler API and submit a new PR.

[mrjustaguy]

it should also be noted that Nvidia's going about their own approach to this problem via Streamline integration into Godot, which can be used to integrate FSR and XeSS and is used to integrate DLSS

[BastiaanOlij]

As far as I know this PR won't be adopted in the current form according to the rendering meeting.
The upscaler system is currently subject to refactor (see #114670). Once that refactor is done, I will hopefully migrate this PR to the new custom upscaler API and submit a new PR.

I've rebased my PR so it's usable again. Would be good to try and port FSR3 to being a GDExtension using that implementation, see what holes we find