fix: accept an empty gzip as a valid input

[sgasho]

related: denoland/deno#29281

enabled gzip decoder to accept an empty gzip.
I added some test cases for this fix at tests/gzip.rs

reproduction

1. clone deno
2. cargo build --config 'patch.crates-io.async-compression.path="../async-compression"'
3. prepare server_hasdata.js & server_nodata.js
4. prepare client.js
5. /path/to/deno-debug --allow-net client.js

server_hasdata.js

import { serve } from "https://deno.land/[email protected]/http/server.ts";

serve(() => {
  const encoder = new TextEncoder();
  const payload = encoder.encode("Hello Deno!");

  const gzipStream = new ReadableStream({
    start(controller) {
      controller.enqueue(payload);
      controller.close();
    },
  }).pipeThrough(new CompressionStream("gzip"));

  return new Response(gzipStream, {
    headers: {
      "Content-Encoding": "gzip",
    },
  });
});

server_nodata.js

Deno.serve((req) => {
    return new Response(new Uint8Array(), {
        headers: {
            'Content-Encoding': 'gzip',
        },
    });
});

client.js

const result = await fetch('http://localhost:8000/').then(r => r.text());
console.log(result);

result

• no errors with server_nodata.js(used to have an "unexpected end of file" error before)

• no errors with server_hasdata.js as before

[NobodyXu]

Thanks! A few feedback for the code

[NobodyXu]

Thanks LGTM!

cc @robjtede can you have a double check please?

[robjtede]

I think it's important to note that this is a degenerate case, and as such may actually break some expectations of other consumers.

When we say "an empty gzip", we actually mean a real gzip file which decompresses to nothing.

What were allowing here is just "an empty file" and pretending that's a valid gzip, which it isn't:

$ touch a.gz
$ gunzip a.gz
gunzip: a.gz: unexpected end of file

Tools like curl do try to be more lenient when the degenerate cases are obvious, such as empty response payloads. I haven't looked up whether it does this check itself or delegates to its decompression library. (I know in Actix Web, we have similar optimizations before calling into to our [de]compression libs.)

I'm torn with async-compression, as a thin wrapper around these libraries, whether we should be putting any extra heuristics in place for this detection (is there any precedent for us doing this?) or whether the responsibility to implement this improvement should be given to Deno's client.

@NobodyXu If we want to accept that responsibility for all our library's consumers then this impl LGTM.

[NobodyXu]

Hmmm...I wonder if other algorithm has the same behavior, or is that just gzip?

for this PR, maybe we can make the behavior opt-in, if it is hard to implement this behavior outside async-compression @sgasho @robjtede ?

[robjtede]

My personal view is that it's a bug in Deno that lead to this request, it shouldn't be trying to decompress something it knows is empty.

[NobodyXu]

Yeah, looking at the original issue seems like deno knows the body is empty before passing stuff to async-compression

[sgasho]

I also thought that the original issue can be solved only modifying deno codes, which turned out to be more difficult(for me) than I thought.

resp.into_body().into_data_stream() (indirectly using sync-compression there) throws the "unexpected end of file" error and I have no idea how to verify if the file is empty before resp.into_body() or resp.into_body().into_data_stream(). (at least, I noticed that trying to read content-length in headers is useless, it almost always returns None)

Of course, most likely there is a way to determine that it is empty before resp.into_body, but unfortunately I am not familiar with Rust yet and have not been able to find it.

https://github.com/denoland/deno/blob/96ab2d3b68721b4e942f2a5e87818bca50ecf13c/ext/fetch/lib.rs#L880-L898

      let body = loop {
        match &mut *reader {
          FetchResponseReader::BodyReader(reader) => break reader,
          FetchResponseReader::Start(_) => {}
        }


        match std::mem::take(&mut *reader) {
          FetchResponseReader::Start(resp) => {
            let stream: BytesStream = Box::pin(
              resp
                .into_body()
                .into_data_stream()
                .map(|r| r.map_err(std::io::Error::other)),
            );
            *reader = FetchResponseReader::BodyReader(stream.peekable());
          }
          FetchResponseReader::BodyReader(_) => unreachable!(),
        }
      };

My personal view is that it's a bug in Deno that lead to this request, it shouldn't be trying to decompress something it knows is empty.

I see, not trying to decompress empty data is an ideal way.
Or, it might be a good idea to make the error message more detailed.
"unexpected end of file" is not actually wrong, but not intuitive. Something like "failed to parse an empty file" is more detailed, client friendly, and less impact on consumers as well.

In the meantime, I'll give it one more try to see if changing the deno alone will solve the problem.

[sgasho]

Summary so far

The choices are

• close this PR
• make error message more detailed
• make this feature as an opt-in
• else