Add UASTC HDR.

[MarkCallow]

Draft of language to add support for UASTC HDR. KDFS and all KDFS references need updating before this can be considered ready for merge.

Draft says to use colorModel KHR_DF_MODEL_UASTC_HDR (= 167) together with vkFormat
VK_FORMAT_ASTC_4x4_SFLOAT_BLOCK (= 1000066000). Any issues with that?

[MarkCallow]

Draft says to use colorModel KHR_DF_MODEL_UASTC_HDR (= 167) together with vkFormat
VK_FORMAT_ASTC_4x4_SFLOAT_BLOCK (= 1000066000). Any issues with that?

@lexaknyazev please let me know your opinion on this as soon as possible. Rich plans to release his code very soon. We need to decide the representation in a .ktx2 file before then.

[MarkCallow]

A version of https://github.com/BinomialLLC/basis_universal/wiki/UASTC-HDR-Texture-Specification-v1.0 needs to be incorporated into the Data Format Specification. That document has two parts: a description of the UASTC HDR subset of ASTC and a description of transcoding to BC6H. I think for simplicity and ease of use whole thing should be incorporated into KDFS but an argument could be made for putting the subset description in KDFS and the transcoding description into the KTX spec.

[lexaknyazev]

the transcoding description into the KTX spec

The unreleased KDFS version of the original UASTC spec contains both the bitstream and the transcoding steps. I think UASTC HDR should follow the same path.

[MarkCallow]

@richgel999 says we need to save in the file the scale used to scale the maximum value from .exr/.hdr down to what BC6H/ASTC can handle, "roughly ~65k". We'll need to add a new standard metadata item for this. He promised me a reference to a similar field in the .exr standard. Once I have that to refer to I'll draft a description of the item.

[lexaknyazev]

The range for BC6H/ASTC HDR is [0.0 … 65504.0], same as non-negative 16-bit floats.

EXR supports three data types: float16, float32, and uint32. Leaving uint32 aside, other two types are signed. It would make sense to store two values: a scale and an offset, to be able to map any source range to UASTC HDR.

HDR (Radiance) range is [0.0 … 1.698e+38]. Although these values are always non-negative, it would still be useful to support both a scale and an offset for more efficient use of the UASTC HDR range.

[lexaknyazev]

So my proposal would be to add a metadata entry with an 8-byte payload containing two float32 values: scale and offset, with the following usage:

• Allowed for all floating-point Vulkan formats.
• When present, the effective texture values are sampled * scale + offset.

It's important to multiply first so that decoders could use a single FMA instruction to apply the scale and the offset at once.

[MarkCallow]

• When present, the effective texture values are sampled * scale + offset

Let me make sure I have this correct. During encoding the "sampled" value will be calculated as sampled = (tv - offset) /scale where tv is the original texture value. Then that value is restored during sampling by tv = sampled * scale + offset. Correct? I realize that in the second case sampled is the filtered value from the texture so "sampled" in my 2 equations are not identical quantities. I'm trying to keep things simple.

[lexaknyazev]

Yes, that's correct. Exact choice of scale and offset is up to the encoder but they must be finite values, i.e., neither NaN nor Inf.

Whether sampled is post- or pre-filtered is an implementation detail. Assuming that original floating-point values are linear and the filtering is linear or nearest (e.g., not cubic) the results would be the same modulo FP precision.

[MarkCallow]

So my proposal would be to add a metadata entry with an 8-byte payload containing two float32 values: scale and offset, with the following usage:

How about KTXmapRange which has the benefit of being a multiple of 4 bytes long (when the terminating NUL is included).

[fluppeteer]

the transcoding description into the KTX spec

The unreleased KDFS version of the original UASTC spec contains both the bitstream and the transcoding steps. I think UASTC HDR should follow the same path.

I was going to email this, but this is as good a place as any...

I agree that describing both is ideal. I've been making decent progress in merging everything for a 1.3.2 release and should have something reviewable over the weekend, but we had stalled at the point of folding in UASTC before, and I wanted clarification on the thing that had previously made me hesitate:

There was a comment that "to get 32-bpp data from a UASTC texture, it's recommended to first transcode to ASTC, then decode that by following the ASTC specification and any applicable extensions", and I don't think I ever had clarification on whether this meant to remove the entire "Decoding process" section (which describes individual pixel colour values). Is that the intent, or should I preserve both, with the note that they're supposed to be identical and that decoding via ASTC is still recommended?

[richgel999]

There was a BinomialLLC/basis_universal#162 (comment) that "to get 32-bpp data from a UASTC texture, it's recommended to first transcode to ASTC, then decode that by following the ASTC specification and any applicable extensions", and I don't think I ever had clarification on whether this meant to remove the entire "Decoding process" section (which describes individual pixel colour values). Is that the intent, or should I preserve both, with the note that they're supposed to be identical and that decoding via ASTC is still recommended?

If you're using BasisU's transcoder, you can transcode UASTC LDR to basist::cTFRGBA32 (or another uncompressed LDR format, like cTFRGB565 etc.). Internally this would skip the ASTC block packing step, and be faster. It will decode the UASTC blocks to a logical format and then immediately decode these block pixels to 32bpp.