TIFF: fix channel unpacking when supplied buffer is too large

[melissalinkert]

Fixes #4058.

Tested using the file linked from #4058 and slightly modified test code:

$ cat BytesTest.java 
import loci.common.DebugTools;
import loci.formats.ImageReader;

class BytesTest {
  public static void main(String[] args) throws Exception {
    DebugTools.enableLogging("WARN");
    var reader = new ImageReader();
    reader.setFlattenedResolutions(false);
    reader.setId("DeltaE_16bit_gamma2_2.tif");
    reader.setResolution(0);

    byte[] correct = reader.openBytes(0, 0, 0, 2048, 2048);

    // minimum 25165824

    int count = 0;
    int start = 25165824;
    int numRuns = Integer.parseInt(args[0]);
    for (int i = start; i < start + numRuns; i++) {
      try {
        byte[] bytes = new byte[i];
        reader.openBytes(0, bytes, 0, 0, 2048, 2048);
        for (int p=0; p<correct.length; p++) {
          if (correct[p] != bytes[p]) {
            System.out.println("incorrect plane @ i=" + i + ", p=" + p);
          }
        }
      } catch (Exception e) {
        if (count == 0) {
          e.printStackTrace();
        }
        System.out.println(i);
        count++;
      }
    }
    System.out.println("total bad: " + count);

  }
}
$ java BytesTest 1000
total bad: 0

That checks for exceptions with large buffers (as in the original test), but also compares the returned bytes against a known-good array without a pre-allocated buffer.

I expect this needs a unit test that uses a TIFF generated from fake data, but am opening first to see if any tests fail overnight.

[melissalinkert]

Test added in b7feb04 is expected to fail without the fix in 1e730ef.

[sbesson]

Tested using the sample files linked from the original issue and a modified version of the script linked in the description to take the TIFF file as a second argument and report the incorrect byte values check as well as exceptions while calling openBytes.

With Bio-Formats 8.5.0, calling the sample files with increasing number of runs starting failing from N=4 for the 16-bit files and N=1 for the 8-bit files

sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads %  for i in $(seq 0 10); do java -cp  bftools/bioformats_package.jar:. BytesTest $i DeltaE_16bit_gamma1.0.tif; done
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 1
total incorrect: 0, total bad: 2
total incorrect: 1, total bad: 2
total incorrect: 2, total bad: 2
total incorrect: 3, total bad: 2
total incorrect: 4, total bad: 2
total incorrect: 4, total bad: 3
sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads %  for i in $(seq 0 10); do java -cp  bftools/bioformats_package.jar:. BytesTest $i DeltaE_16bit_gamma2.2.tif; done 
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 1
total incorrect: 0, total bad: 2
total incorrect: 1, total bad: 2
total incorrect: 2, total bad: 2
total incorrect: 3, total bad: 2
total incorrect: 4, total bad: 2
total incorrect: 4, total bad: 3
sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads %  for i in $(seq 0 10); do java -cp  bftools/bioformats_package.jar:. BytesTest $i DeltaE_8bit_gamma1.0.tif; done
total incorrect: 0, total bad: 0
total incorrect: 1, total bad: 0
total incorrect: 2, total bad: 0
total incorrect: 3, total bad: 0
total incorrect: 4, total bad: 0
total incorrect: 5, total bad: 0
total incorrect: 6, total bad: 0
total incorrect: 7, total bad: 0
total incorrect: 8, total bad: 0
total incorrect: 9, total bad: 0
total incorrect: 10, total bad: 0
sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads %  for i in $(seq 0 10); do java -cp  bftools/bioformats_package.jar:. BytesTest $i DeltaE_8bit_gamma2.2.tif; done 
total incorrect: 0, total bad: 0
total incorrect: 1, total bad: 0
total incorrect: 2, total bad: 0
total incorrect: 3, total bad: 0
total incorrect: 4, total bad: 0
total incorrect: 5, total bad: 0
total incorrect: 6, total bad: 0
total incorrect: 7, total bad: 0
total incorrect: 8, total bad: 0
total incorrect: 9, total bad: 0
total incorrect: 10, total bad: 0

With this PR included,

sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads % for i in $(seq 0 10); do java -cp  ~/Documents/GitHub/bioformats/artifacts/bioformats_package.jar:. BytesTest $i DeltaE_16bit_gamma1.0.tif; done
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads % for i in $(seq 0 10); do java -cp  ~/Documents/GitHub/bioformats/artifacts/bioformats_package.jar:. BytesTest $i DeltaE_16bit_gamma2.2.tif; done 
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads % for i in $(seq 0 10); do java -cp  ~/Documents/GitHub/bioformats/artifacts/bioformats_package.jar:. BytesTest $i DeltaE_16bit_gamma1.0.tif; done 
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
sbesson@Sebastien-GS-MacBook-Pro-2025 Downloads % for i in $(seq 0 10); do java -cp  ~/Documents/GitHub/bioformats/artifacts/bioformats_package.jar:. BytesTest $i DeltaE_16bit_gamma2.2.tif; done 
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0
total incorrect: 0, total bad: 0

Also that the new added unit tests fail with an ArrayIndexOutOfBoundsException with 1e730ef reverted but pass with it included.