Skip to content

Lack of deflate backwards compatibility? #72

@Lupus

Description

@Lupus

I've faced issues with standard linux unzip being unable to unzip the file that I create programmatically with xflate compression. I decided to repro on a smaller case and got same results. I basically took the example from the documentation of xflate package and outputted the resulting zip to a file, source code follows:

package main

import (
	"archive/zip"
	"io"
	"io/ioutil"
	"log"
	"os"

	"github.com/dsnet/compress/xflate"
)

func init() { log.SetFlags(log.Lshortfile) }

// MustLoadFile must load a file or else panics.
func MustLoadFile(file string) []byte {
	b, err := ioutil.ReadFile(file)
	if err != nil {
		panic(err)
	}
	return b
}

func main() {
	// Test files of non-trivial sizes.
	files := map[string][]byte{
		"twain.txt":   MustLoadFile("testdata/twain.txt"),
		"digits.txt":  MustLoadFile("testdata/digits.txt"),
		"huffman.txt": MustLoadFile("testdata/huffman.txt"),
	}

	// Write the Zip archive.
	out, err := os.Create("output.zip")
	if err != nil {
		log.Fatal(err)
	}
	zw := zip.NewWriter(out)
	zw.RegisterCompressor(zip.Deflate, func(wr io.Writer) (io.WriteCloser, error) {
		// Instead of the default DEFLATE compressor, register one that uses
		// XFLATE instead. We choose a relative small chunk size of 64KiB for
		// better random access properties, at the expense of compression ratio.
		return xflate.NewWriter(wr, &xflate.WriterConfig{
			Level:     xflate.BestSpeed,
			ChunkSize: 1 << 16,
		})
	})
	for _, name := range []string{"twain.txt", "digits.txt", "huffman.txt"} {
		body := files[name]
		f, err := zw.Create(name)
		if err != nil {
			log.Fatal(err)
		}
		if _, err = f.Write(body); err != nil {
			log.Fatal(err)
		}
	}
	if err := zw.Close(); err != nil {
		log.Fatal(err)
	}
	err = out.Close()
	if err != nil {
		log.Fatal(err)
	}
}

Here's what's happening with output.zip that this program created:

$ unzip -l output.zip
Archive:  output.zip
  Length      Date    Time    Name
---------  ---------- -----   ----
   387969  1980-00-00 00:00   twain.txt
   100003  1980-00-00 00:00   digits.txt
   262144  1980-00-00 00:00   huffman.txt
---------                     -------
   750116                     3 files

$ unzip -t output.zip
Archive:  output.zip
    testing: twain.txt               (incomplete d-tree)
  error:  invalid compressed data to inflate
    testing: digits.txt              (incomplete d-tree)
  error:  invalid compressed data to inflate
    testing: huffman.txt             (incomplete d-tree)
  error:  invalid compressed data to inflate
At least one error was detected in output.zip.

I'm not sure that's an expected behavior. To double check I've took the zip to my Windows system, and it was also unable to extract the files.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions