Bounds-check decompression against corrupted input

[alexey-milovidov]

Problem

The decompression path reads sizes, counts and dimensions from the compressed stream and uses them to drive reads and writes without validating them against the size of the input buffer. On corrupted or adversarial input (for example when fuzzing the decompressor) this leads to out-of-bounds reads and writes — a decode() loop bounded only by the target length, a Huffman tree reconstructed from an unvalidated node count, an unchecked payload-size field in SZ_decompress, etc.

Changes

This adds bounds checks along the decompression path. Valid data is unaffected — the checks only fire on input that would otherwise read or write out of bounds.

• MemoryUtil: the length-checked read() overloads (the ones that take remaining_length) now throw std::out_of_range instead of assert(), which is a no-op in release builds.

• SZ_decompress (sz.hpp): validate that the buffer is at least the 16-byte header and that the stored compressed-payload size fits in the remaining buffer.

• Config::load: take the size of the config blob and bound every read against it; validate the dimension count (1..4) and per-dimension bit width before reading the dimensions; and check that the product of the dimensions equals the element count. (This also includes the end-pointer fix from Fix Config::load reading one byte past the end of the config #131.)

• HuffmanEncoder::load/decode: validate the node count, the serialized tree size and the encoded length against the available bytes, and guard against null nodes produced by a malformed tree.

• HuffmanEncoder tree reconstruction: unpad_tree now rejects child indices that are out of range (a valid tree always assigns a child a higher index than its parent, so parent < child < nodeCount), which prevents out-of-bounds reads of the L/R/C/t arrays and cycles; and a stateNum too small to hold nodeCount nodes is rejected so the node pool can not overflow.

• LinearQuantizer: recover_unpred bounds its index against the unpredictable-value vector, and load validates the unpredictable count against the remaining bytes before resizing.

• RegressionPredictor / ComposedPredictor: bound the per-block coefficient/selection indices against the decoded vectors, and validate the selected predictor id.

• Lossless_zstd::decompress / SZ_decompress_dispatcher: on the ALGO_LOSSLESS path the pre-allocated output buffer is now passed to the lossless decoder with its capacity, and a payload that declares a larger decompressed size is rejected before zstd writes — previously the size came from the payload and was used as the zstd destination capacity unchecked (a heap buffer overflow).

• SZGenericCompressor::decompress: read the quantization-index count with the bounded overload so a truncated buffer is not read past its end.

• SZ_decompress_OMP (the OpenMP decompression path): bound the thread-count read, parse each per-thread config with the bounded Config::load, bound the per-thread size array, and build the per-thread payload offsets with an overflow-safe bound so each thread's slice stays within the buffer.

• Generic lossy path (SZGenericCompressor::decompress): the internal buffer size is read from the payload and was allocated unbounded for ALGO_INTERP/ALGO_LORENZO_REG. It is now bounded by the largest buffer the configuration could have produced (4096 + conf.size_est() + ZSTD_compressBound(conf.num * sizeof(T)), valid because ZSTD_compressBound(B) >= B), passed as a capacity that Lossless_zstd::decompress enforces before allocating.

• Lossless_zstd::decompress fail-closed: also require ZSTD_decompress to produce exactly the declared size, so a frame that expands to fewer bytes can not leave the tail of the (caller-provided) output buffer uninitialized.

• Inner interpolation dimensions + RAII scratch buffer: InterpolationDecomposition::decompress rejects a block whose separately-stored dimensions do not match the trusted config (otherwise it would iterate past the output buffer), and SZGenericCompressor::decompress owns its scratch buffer with RAII so a corrupted block can not leak it.

• Pointer-overflow UB + Huffman over-read (sanitizer findings): LorenzoPredictor neighbour helpers computed d[-offset] with an unsigned offset (*(d + (size_t)(-offset)), a wrapped/out-of-bounds pointer flagged by -fsanitize=pointer-overflow) — now use pointer subtraction; and the quant_inds count is written before the encoder so its serialized tree is immediately followed by its stream, so the decode bound recorded by load() is no longer one field too large (which let a truncated block over-read).

Notes

• Config::load gains a second overload load(const unsigned char*&, size_t cmpSize); the existing one-argument overload is kept for internal/trusted callers and delegates with an unbounded size.
• This overlaps Fix Config::load reading one byte past the end of the config #131 in Config::load (it incorporates the same end-pointer fix). If Fix Config::load reading one byte past the end of the config #131 is merged first this rebases trivially.
• This does not change the value of any successfully-decompressed data; the new checks only reject inputs that are inconsistent with the buffer.

Context

Found while integrating SZ3 as an experimental compression codec in ClickHouse and fuzzing the decompressor: ClickHouse/ClickHouse#108788