Skip to content

Releases: dfa1/zstd-java

v0.10

Choose a tag to compare

@github-actions github-actions released this 18 Jul 18:03

Added

  • Native builds now compile with ZSTD_MULTITHREAD, making
    ZstdCompressParameter.NB_WORKERS (plus JOB_SIZE and OVERLAP_LOG)
    functional instead of a silent no-op. Workers engage above zstd's 512 KiB
    job-size minimum; multithreaded frames are format-valid but not
    byte-identical to single-threaded output. A context that compressed with
    workers holds its native worker threads until close()reset() does
    not release them, so give such contexts a dedicated owner (never pool
    them). See ADR 0015, which
    supersedes ADR 0014.
    (#80)

v0.9

Choose a tag to compare

@github-actions github-actions released this 18 Jul 07:33

Added

  • Native builds now decode legacy zstd frame formats v0.4-v0.7
    (ZSTD_LEGACY_SUPPORT=4, matching zstd-jni's default). v0.1-v0.3 stay
    unsupported — they predate zstd's 1.0 stabilization and are essentially
    never seen in practice. Verified against a real fixture of five concatenated
    legacy frames extracted from zstd's own test suite. (#73)

Changed

  • linux-x86_64/osx-x86_64/windows-x86_64 native builds now include
    zstd's hand-written BMI2 Huffman-decode assembly (previously disabled). It
    is a no-op on non-x86_64 targets and only activates via zstd's own runtime
    CPU detection; benchmarked as throughput-neutral on this project's
    synthetic workload but carries no measured downside either.
    (#71)
  • aarch64 native builds now target an ARMv8-A + CRC baseline
    (-mcpu=generic+crc, zig's spelling of -march=armv8-a+crc), instead of
    the fully generic baseline. Measured +6.9% compress / +12-14% decompress
    throughput on Apple Silicon. (#71)

Security

  • linux-x86_64/linux-aarch64 native builds now link with full RELRO and
    immediate binding (-Wl,-z,relro,-z,now), closing off the classic
    GOT-overwrite exploit primitive. Verified with llvm-readelf.
    (#71)
  • windows-x86_64/windows-aarch64 native builds now export only zstd's
    public API (ZSTD_*/ZDICT_*) from the DLL, via -DZSTD_DLL_EXPORT=1
    (the PE analogue of the -fvisibility=hidden surface already used on
    ELF/Mach-O), instead of dumping every internal symbol
    (FSE_*/HUF_*/COVER_*/...) into the export table via
    --export-all-symbols. Cut the windows-x86_64 export table from 576 to
    185 symbols. (#79)

Fixed

  • Building the native library from source on Windows was silently broken:
    Maven's exec plugin tried to execute build-zstd.sh directly, which only
    works via a shebang on macOS/Linux. Windows builds now invoke it through
    bash explicitly. A second latent bug this surfaced — unrecognized/Windows
    host OS detection crashed the build script under set -u — is fixed
    alongside it. (#75)
  • Native library compilation now runs every translation unit through a real
    parallel work queue (xargs -P) instead of a fixed-size batch-then-wait
    loop, and aborts immediately if any zig cc invocation fails. Previously a
    failed compile under &/wait was invisible to set -e — it could
    silently produce no .o and only surface later as a cryptic link error, or
    worse, a link that "succeeded" against a stale .o left over from a
    previous run. (#78)

Investigated and rejected as part of the same effort (see
#70 for full benchmark data):
LTO (real compress regression on x86_64, unsupported on macOS entirely — zig's
Mach-O linker has no LTO support, tracked in
#77) and an x86-64-v3
baseline (mixed result, hurts compress more than it helps decompress). Both
would have traded away this project's existing compress-side edge over
zstd-jni for a smaller decompress-side gain.

v0.8

Choose a tag to compare

@github-actions github-actions released this 14 Jul 06:02

Added

  • module-info.java: zstd now ships as a named JPMS module
    (module io.github.dfa1.zstd), exporting the single public API package.
    Module-path consumers grant --enable-native-access=io.github.dfa1.zstd
    instead of ALL-UNNAMED; classpath consumers are unaffected. See
    ADR 0011.

v0.7

Choose a tag to compare

@github-actions github-actions released this 28 Jun 12:16

Changed

  • Breaking: renamed public types to spell out abbreviations, matching the
    Zstd<Compress|Decompress><Stream|Parameter> family and zstd's own prose
    ("compression context", "dictionary"): ZstdCompressCtxZstdCompressContext,
    ZstdDecompressCtxZstdDecompressContext, ZstdCompressDict
    ZstdCompressDictionary, ZstdDecompressDictZstdDecompressDictionary.

v0.6

Choose a tag to compare

@github-actions github-actions released this 27 Jun 16:57
99e7dd6

Added

  • ZstdCompressCtx.refPrefix(MemorySegment) / ZstdDecompressCtx.refPrefix(...)
    — reference native content as a single-use prefix (raw-content dictionary) for
    the next frame only: the building block for delta compression (compress a new
    version against a similar previous one). The prefix is referenced, not copied
    or digested, and writes no dictionary ID; the decompressor must reference the
    same prefix to decode. Binds ZSTD_CCtx_refPrefix / ZSTD_DCtx_refPrefix.
    Segment-only by design — heap callers that need a copy should use
    loadDictionary instead.
  • Zstd.dictId(byte[]) / Zstd.dictId(MemorySegment) — read the dictionary id
    stamped in raw dictionary bytes without wrapping them in a ZstdDictionary.
    Binds ZSTD_getDictID_fromDict.
  • ZstdDictionaryId value type — a record wrapping the 32-bit dictionary id
    with an unsigned value(), isPresent(), and the NONE sentinel for "no id".
  • ZstdFrame.decompressedSize(byte[]) / ZstdFrame.decompressedSize(MemorySegment)
    — the exact combined decompressed size of all concatenated frames, summed from
    each frame header (throws if any frame does not record its size). Complements
    decompressedBound (upper bound). Binds ZSTD_findDecompressedSize.
  • ZstdFrame.headerSize(byte[]) / ZstdFrame.headerSize(MemorySegment) — the size
    of a frame's header computed from just its leading bytes (as few as 5), without a
    full parse. Binds ZSTD_frameHeaderSize.
  • ZstdFrame.decompressionMargin(byte[]) / ZstdFrame.decompressionMargin(MemorySegment)
    — the extra room needed to decompress a frame in place (output buffer overlaps
    the compressed input at its tail), sized decompressedSize + margin. Binds
    ZSTD_decompressionMargin.
  • ZstdDictionary.compressDict(int) / compressDict() / decompressDict()
    factories for digested dictionaries, e.g. dict.compressDict(19) instead of
    new ZstdCompressDict(dict, 19). They signal that the result is AutoCloseable
    and are for sharing one digest across contexts via refDictionary; a single
    context should prefer the context-owned loadDictionary.

Changed

  • Every dictionary-id accessor now returns ZstdDictionaryId instead of int:
    ZstdDictionary.id(), ZstdCompressDict.id(), ZstdDecompressDict.id(),
    ZstdFrame.dictId(...), and ZstdFrameHeader.dictId(). The 0 sentinel is now
    ZstdDictionaryId.NONE, and the id reads as unsigned via value().
  • Zstd.decompress(byte[]) now throws ZstdException (instead of letting a raw
    ArithmeticException escape) when a frame declares a content size larger than a
    Java array can hold. The size comes from the untrusted frame header; use
    decompress(byte[], int) to bound output for untrusted input.

v0.5

Choose a tag to compare

@github-actions github-actions released this 27 Jun 07:07

Added

  • ZstdCompressCtx.reset(ZstdResetDirective) / ZstdDecompressCtx.reset(...)
    recycle a context's native state between frames without freeing and recreating
    it. SESSION_ONLY keeps the level, parameters, and dictionary; PARAMETERS /
    SESSION_AND_PARAMETERS restore the defaults. Binds ZSTD_CCtx_reset /
    ZSTD_DCtx_reset.
    (3dfd5b8)
  • ZstdCompressCtx.loadDictionary(...) / ZstdDecompressCtx.loadDictionary(...)
    (a ZstdDictionary or a native MemorySegment) and refDictionary(...) (a
    pre-digested ZstdCompressDict / ZstdDecompressDict, attached by reference,
    no copy). A sticky dictionary on the context lets compression combine a
    dictionary with the advanced parameters (checksum, window log, long-distance
    matching) — impossible through the per-call compress(src, dict) overloads,
    which route the legacy dictionary path. A parameter reset(...) clears it.
    Binds ZSTD_CCtx_loadDictionary / ZSTD_DCtx_loadDictionary (now on contexts,
    not just streams), ZSTD_CCtx_refCDict, ZSTD_DCtx_refDDict.
    (3dfd5b8)

Changed

  • NativeLibrary.classifier() now throws a clear UnsatisfiedLinkError naming
    the unsupported CPU arch instead of silently mapping it to x86_64 (which
    deferred failure to a cryptic dlopen error). Added an explicit amd64
    branch so Linux JVMs (which report os.arch=amd64) still resolve x86_64.
    (ea1ac84)

Fixed

  • Native JARs are much smaller. The ELF shared library is now stripped at link
    time (-s), dropping debug info (libzstd.so 4.0M -> ~650K), and the
    multi-MB .pdb debug database and .lib import library that lld emits next
    to the Windows .dll are no longer bundled (neither is needed at runtime).
    Net: linux-x86_64 native jar 1.2M -> 285K, windows-x86_64 1.2M -> 372K.
    (ea1ac84)

v0.4

Choose a tag to compare

@github-actions github-actions released this 26 Jun 21:01

Added

  • Zstd.versionNumber() — the linked zstd version as a single integer
    (MAJOR * 10000 + MINOR * 100 + PATCH, e.g. 10507 for 1.5.7), for
    programmatic version checks alongside version().

Changed

  • ZstdSkippableContent is now a true immutable value: it defensively copies its
    bytes on the way in and out, and compares by content
    (equals / hashCode / toString over the payload, not array identity).
  • Public methods fail fast with a named NullPointerException on null byte[],
    dictionary, or sample arguments, instead of an opaque failure deep in native
    code. Streams are documented as not thread-safe; digested dictionaries
    (ZstdCompressDict / ZstdDecompressDict) as immutable and safe to share.

Fixed

  • A streaming wrapper that failed partway through construction (e.g. an invalid
    parameter or dictionary) leaked the native context. The context is now freed
    on every constructor error path.

Security

  • The bundled library is extracted into a directory created owner-only
    (rwx------) atomically at creation, not just by default. The third-party
    setup-zig CI action is pinned to a full commit SHA.

v0.3

Choose a tag to compare

@github-actions github-actions released this 26 Jun 16:33

Changed

  • zstd-platform is now an ordinary (empty) jar instead of a pom aggregator:
    depend on it like any other artifact — drop the <type>pom</type> you needed
    before. It still transitively pulls the bindings plus all six native libraries.

Security

  • Native loading is bundled-only. Removed the -Dzstd.lib.path override —
    loading a caller-supplied native library is arbitrary native code execution in
    the JVM, so the loader now trusts only the artifact bundled on the classpath.
    The bundled library is extracted into a private, owner-only temp directory
    (closing a swap/symlink window in the shared temp root). To run a self-built
    libzstd, rebuild it into the native resource jar (see docs/how-to.md).

Fixed

  • ZstdSkippableContent now compares by content: equals / hashCode /
    toString consider the payload bytes instead of array identity.

v0.2

Choose a tag to compare

@github-actions github-actions released this 26 Jun 13:58

Added

  • zstd-platform aggregator artifact: one dependency pulls the bindings plus
    every platform's native library, so a build runs on any OS/arch without
    choosing a classifier.
  • MemorySegment constructors on ZstdCompressDict / ZstdDecompressDict to
    digest a dictionary straight from off-heap memory (e.g. an mmap slice) with no
    heap byte[] copy.

v0.1

Choose a tag to compare

@github-actions github-actions released this 26 Jun 13:03

First release. Java 25 Foreign Function & Memory (FFM) bindings for
Zstandard 1.5.7, built hermetically from
vendored source with zig cc (no JNI, no prebuilt binaries). 68 of the public
zstd symbols are bound; see docs/supported.md.

Added

  • One-shot compression/decompression over byte[] and zero-copy MemorySegment
    (Zstd, ZstdCompressCtx, ZstdDecompressCtx).
  • Dictionaries: training (ZDICT_trainFromBuffer, COVER / fast-COVER optimisers,
    finalizeDictionary), digested ZstdCompressDict / ZstdDecompressDict,
    dictionary ids and header size.
  • Streaming: ZstdOutputStream / ZstdInputStream (java.io) and a zero-copy
    MemorySegment driver (ZstdCompressStream / ZstdDecompressStream), with
    dictionaries, pledgedSrcSize, and live progress().
  • All advanced parameters (ZstdCompressParameter / ZstdDecompressParameter)
    with bounds queries; checksum, long-distance matching, window log, etc.
  • Frame inspection (ZstdFrame): header, content/compressed size, dictionary id,
    skippable frames.
  • Typed errors (ZstdException.code() / ZstdErrorCode) and memory accounting
    (sizeOf(), Zstd.estimate*Size).
  • Native artifacts for macOS, Linux and Windows on x86_64 and aarch64,
    cross-compiled from a single host with zig cc.
  • Format-compatibility tests against the reference zstd-jni binding.