0.59.2

Changes

🧰 Maintenance

• Add ci check for java publish (#6578) @robert3005

0.59.1

Changes

🐛 Bug Fixes

• fix: CUDA scalar inplace (#6567) @0ax1

🧰 Maintenance

• chore: drop flaky duckdb http e2e test (#6570) @0ax1
• chore: make auto clang-format consistent (#6568) @0ax1

0.59.0

Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better vx-bench docs in README (#5900) @AdamGS

⚠️ Breaks

• Remove vortex-compute (#6561) @connortsui20
• Introduce VortexError::Other and simplify Display implementation (#6410) @AdamGS
• Move vortex-scalar into vortex-array (#6523) @connortsui20
• break[array]: lazy between (#6528) @joseph-isaacs
• break[array]: mask compute to lazy mask execution (#6485) @joseph-isaacs
• break[array]: lazy compare function (#6495) @joseph-isaacs
• Remove vortex-vector (#6529) @connortsui20
• break[array]: move like compute to lazy compute (#6497) @joseph-isaacs
• Pass buffers to VTable::deserialize (#6473) @connortsui20
• break[array]: move cast traits to exprs (#6487) @joseph-isaacs
• Revert #6363 (#6476) @connortsui20
• Use Scalar directly for constant array metadata inlining (#6439) @connortsui20
• Fix Scalar de/serialization with proto and outline nulls from ScalarValue (#6309) @connortsui20
• chore[array]: inline take compute function (#6376) @joseph-isaacs
• Add Array::deserialize arguments (#6340) @gatesn
• Update FSST to store components of compressed codes (#6325) @gatesn
• Array Execution (#6307) @gatesn
• Execute parent to return ArrayRef (#6257) @gatesn
• Extension DType vtables (#6081) @gatesn
• break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
• Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
• fallible scalar_at and validity (#6102) @joseph-isaacs
• Remove the python feature from vortex-error (#6064) @AdamGS
• break[*]: make array methods fallible (#6086) @joseph-isaacs
• chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
• Array VTable (#6000) @gatesn
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
• Remove EncodeVTable (#6036) @gatesn
• chore: fallible canonical (#6006) @joseph-isaacs
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs

✨ Features

• Add FileSystem trait and make DuckDB configurable (#6564) @gatesn
• Add scan concurrency config for datafusion source (#5945) @AdamGS
• feat: implement DuckDB filesystem integration for Vortex file handling (#6198) @iceboundrock
• vx convert to support cuda compatible vortex files (#6522) @onursatici
• cuda flat layout (#6521) @onursatici
• feat[xtask]: faster public-api (#6498) @joseph-isaacs
• Improve substrait support to include more types and expressions (#6484) @AdamGS
• feat[array]: refactor zip compute kernel to lazy zip via ScalarFn Array (#6461) @joseph-isaacs
• feat[gpu]: CUDA dynamic dispatch kernel (#6483) @0ax1
• feat[array]: lazy numeric (#6469) @joseph-isaacs
• feat[array]: move over to a expr based cast (#6404) @joseph-isaacs
• refactor[array]: move boolean ops to lazy binary expr (#6447) @joseph-isaacs
• Public api lockfiles (#6348) @gatesn
• feat[array]: refactor invert compute kernel to lazy not (#6446) @joseph-isaacs
• sqllogictest runner for DF and DuckDB on Vortex (#6273) @AdamGS
• feat[array]: move over fill_null to scalar_fn array making it lazy (#6406) @joseph-isaacs
• Upgrade Pco to 1.0.1 (#6354) @mwlon
• Add execution target that recursively canonicalises as well as canonical with canonical validity (#6380) @robert3005
• dict of strings kernel to support outlined views (#6401) @onursatici
• zstd buffers cuda kernel (#6394) @onursatici
• Make VortexSource::new public (#6392) @AdamGS
• Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
• feat[expr]: new fill null expr (#6385) @joseph-isaacs
• zstd buffers array (#6362) @onursatici
• gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
• cuda dictionary of vbv decompression (#6361) @0ax1
• CLI flag to enable tracing for benchmarks (#6350) @gatesn
• chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
• chore[cuda]: add date32 exporter test (#6345) @a10y
• feat: display tree with no metadata (#5898) @danking
• feat[cuda]: device exports for varbinview (#6338) @a10y
• vortex-metrics v2 (#6187) @AdamGS
• feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
• feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
• feat[gpu]: CUDA date_time_parts kernel (#6303) @0ax1
• feat: alternate for StructFields (#6276) @blaginin
• Add a canonical matcher (#6292) @gatesn
• Move over to array matchers (#6291) @gatesn
• feat[gpu]: cuda runend decoding (#6275) @0ax1
• feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
• feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
• feat[array]: validate array encodings on write (#6241) @joseph-isaacs
• feat[cuda]: SequenceArray (#6232) @a10y
• feat[cuda]: patches kernel (#6231) @a10y
• Pass limit through Python Vortex (#6239) @gatesn
• Ship Vortex CLI as part of Python package (#6233) @gatesn
• Feature: Add more Filter<_> implementations (#6173) @connortsui20
• Benchmarks website revamped (#6215) @palaska
• Dataset API (#5949) @gatesn
• feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
• feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
• feat[python]: support full read/write from object storage (#6022) @a10y
• feat[scan]: gpu scan (#6199) @joseph-isaacs
• feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
• feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
• Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
• feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
• Feature: Use array.filter() instead of filter(array) compute (#6146) @connortsui20
• Rewrite benchmark website (#6160) @palaska
• feat: stats pruning for LIKE filters (#6049) @a10y
• feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
• feat[cuda]: canonical to_host (#6130) @joseph-isaacs
• feat[cuda]: dict decimal take (#6129) @joseph-isaacs
• feat[gpu]: dict take values prim (#6101) @joseph-isaacs
• chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
• feat[gpu]: scalar encodings (#6109) @joseph-isaacs
• Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
• feat: nvcomp wrapper crate (#6108) @0ax1
• Allow configuring different readers for DF integration (#6027) @AdamGS
• feat: introduce CudaDeviceBuffer (#6060) @0ax1
• Array VTable (#6000) @gatesn
• feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
• Attempting to simplify the IO read traits (#5935) @AdamGS
• feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
• feat[array]: execute::<PrimitiveArray>() (#5954) @joseph-isaacs
• Spark 4 (#5944) @a10y
• feat: cuda gpu skeleton (#5936) @0ax1
• Add flag to run benchmarks under Samply (#5940) @gatesn
• Datafusion 52 upgrade (#5863) @AdamGS
• Support precision casting for DScalar (#5937) @AdamGS
• feat[array]: use execute in canonical (#5925) @joseph-isaacs
• feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs
• feature: TableStrategy (#5640) @a10y

🚀 Performance

• Faster primitive take (#6559) @connortsui20
• perf[vortex-btrblocks]: remove listview expansion ratio (#6530) @asubiotto
• VarBinBuilder: eagerly set IsSorted stat on offsets (#6452) @dimitarvdimitrov
• Implement ReduceNode for Array to not require creating new arc when returning children (#6443) @robert3005
• chore: clean cancelled read requests earlier in driver state transitions (#6382) @caicancai
• shared array to avoid duplicate execute (#6453) @onursatici
• vortex-btrblocks: write listviews when elements expand > 50% (#6393) @asubiotto
• Implement ReduceNode for ScalarFnArray to avoid boxing (#6435) @robert3005
• Improve visitor performance by reducing Vec and String allocations (#6337) @AdamGS
• lazy dict layout values eval (#6423) @onursatici
• cuda strategy to use zstd buffers for strings (#6403) @onursatici
• bitunpacking cuda kernels store output into shared memory before copying to main memory (#6384) @robert3005
• chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
• constant array metadata for small scalars (#6363) @onursatici
• chore[array]: move over filtering to {reduce/execute}_parent (#6312) @joseph-isaacs
• perf[buf]: inline collect_bool (#6313) @joseph-isaacs
• perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
• chore: list contains over scalars (#6242) @blaginin
• perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
• performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
• perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
• Check read-only cache first (#6249) @gatesn
• Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
• Perf: Use optimized filter implementations is FilterArray::execute (#6189) @connortsui20
• coalesced request align all segments (#6158) @onursatici
• perf: CUDA async to host copy (#6085) @0ax1
• feat: make host to device copy async (#6075) @0ax1
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
• Only check for ConstantVTable (#6002) @gatesn
• do not validate is_sorted on runend (#6025) @onursatici
• perf: CUDA FoR loop unrolling (#6017) @0ax1
• feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
• Remove unnecessary work from ListArray::validate (#6004) @AdamGS
• scan: make LazyS...

0.59.0-rc.0

Changes

• docs: tidy and unify Python quickstart examples (#5972) @caicancai
• Better vx-bench docs in README (#5900) @AdamGS

⚠️ Breaks

• chore[array]: inline take compute function (#6376) @joseph-isaacs
• Add Array::deserialize arguments (#6340) @gatesn
• Update FSST to store components of compressed codes (#6325) @gatesn
• Array Execution (#6307) @gatesn
• Execute parent to return ArrayRef (#6257) @gatesn
• Extension DType vtables (#6081) @gatesn
• break[expr]: remove Expression.evaluate use Array.apply(expr) instead (#6167) @joseph-isaacs
• Make metrics more local to prevent unbounded memory usage (#6156) @AdamGS
• fallible scalar_at and validity (#6102) @joseph-isaacs
• Remove the python feature from vortex-error (#6064) @AdamGS
• break[*]: make array methods fallible (#6086) @joseph-isaacs
• chore[array]: replace to_canonical with to execute. (#6061) @joseph-isaacs
• Array VTable (#6000) @gatesn
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• chore[array]: remove CanonicalVTable and move append_to_builder to main VTable (#6045) @joseph-isaacs
• Remove EncodeVTable (#6036) @gatesn
• chore: fallible canonical (#6006) @joseph-isaacs
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs

✨ Features

• Make VortexSource::new public (#6392) @AdamGS
• Make projection pushdown in DF an opt-in feature (#6373) @AdamGS
• feat[expr]: new fill null expr (#6385) @joseph-isaacs
• zstd buffers array (#6362) @onursatici
• gpu compatible write strategy, move compact strategy to use btrblocks with zstd and pco (#6322) @onursatici
• cuda dictionary of vbv decompression (#6361) @0ax1
• CLI flag to enable tracing for benchmarks (#6350) @gatesn
• chore[array]: move take kernel to execute/reduce rules (#6310) @joseph-isaacs
• chore[cuda]: add date32 exporter test (#6345) @a10y
• feat: display tree with no metadata (#5898) @danking
• feat[cuda]: device exports for varbinview (#6338) @a10y
• vortex-metrics v2 (#6187) @AdamGS
• feat[array]: dict array allows signed codes (#6336) @joseph-isaacs
• feat[cuda]: export arrays to ArrowDeviceArray (#6253) @a10y
• feat[gpu]: CUDA date_time_parts kernel (#6303) @0ax1
• feat: alternate for StructFields (#6276) @blaginin
• Add a canonical matcher (#6292) @gatesn
• Move over to array matchers (#6291) @gatesn
• feat[gpu]: cuda runend decoding (#6275) @0ax1
• feat[btrblocks]: add a dynamic btr blocks compressor (#6207) @joseph-isaacs
• feat[cuda]: constant array for numeric types (#6248) @joseph-isaacs
• feat[array]: validate array encodings on write (#6241) @joseph-isaacs
• feat[cuda]: SequenceArray (#6232) @a10y
• feat[cuda]: patches kernel (#6231) @a10y
• Pass limit through Python Vortex (#6239) @gatesn
• Ship Vortex CLI as part of Python package (#6233) @gatesn
• Feature: Add more Filter<_> implementations (#6173) @connortsui20
• Benchmarks website revamped (#6215) @palaska
• Dataset API (#5949) @gatesn
• feat[cuda]: filter for Decimal/VarBinView (#6196) @a10y
• feat[cuda]: add a cuda stream pool (#6205) @joseph-isaacs
• feat[python]: support full read/write from object storage (#6022) @a10y
• feat[scan]: gpu scan (#6199) @joseph-isaacs
• feat[gpu]: nvidia cub filter kernel (#6188) @0ax1
• feature: Add fastlanes bit unpacking cuda kernels (#6145) @robert3005
• Use built-in DF cache for file metadata and fix metrics wiring (#6164) @AdamGS
• feat: reduce size of segments containing fixed-size-list arrays (#6180) @danking
• Feature: Use array.filter() instead of filter(array) compute (#6146) @connortsui20
• Rewrite benchmark website (#6160) @palaska
• feat: stats pruning for LIKE filters (#6049) @a10y
• feat: Zstd CUDA nvCOMP decompression (#6136) @0ax1
• feat[cuda]: canonical to_host (#6130) @joseph-isaacs
• feat[cuda]: dict decimal take (#6129) @joseph-isaacs
• feat[gpu]: dict take values prim (#6101) @joseph-isaacs
• chore[cuda]: decimal_byte_parts (#6118) @joseph-isaacs
• feat[gpu]: scalar encodings (#6109) @joseph-isaacs
• Add VortexScanReplacement to support reading from '*.vortex' (#6111) @robert3005
• feat: nvcomp wrapper crate (#6108) @0ax1
• Allow configuring different readers for DF integration (#6027) @AdamGS
• feat: introduce CudaDeviceBuffer (#6060) @0ax1
• Array VTable (#6000) @gatesn
• feat[arary]: add a slice array to replace the slice op. (#5958) @joseph-isaacs
• Attempting to simplify the IO read traits (#5935) @AdamGS
• feat: CUDA compilation infra & FoR kernel (#5966) @0ax1
• feat[array]: execute::<PrimitiveArray>() (#5954) @joseph-isaacs
• Spark 4 (#5944) @a10y
• feat: cuda gpu skeleton (#5936) @0ax1
• Add flag to run benchmarks under Samply (#5940) @gatesn
• Datafusion 52 upgrade (#5863) @AdamGS
• Support precision casting for DScalar (#5937) @AdamGS
• feat[array]: use execute in canonical (#5925) @joseph-isaacs
• feat(vx): add JSON output, segments command, and SQL query support (#5642) @palaska
• feat[array]: use execute to return Canonical array not Vector (#5895) @joseph-isaacs
• feature: TableStrategy (#5640) @a10y

🚀 Performance

• chore:optimize next_coalesced() by tracking removed ids in a HashSet (#6353) @caicancai
• constant array metadata for small scalars (#6363) @onursatici
• chore[array]: move over filtering to {reduce/execute}_parent (#6312) @joseph-isaacs
• perf[buf]: inline collect_bool (#6313) @joseph-isaacs
• perf[vortex-array]: avoid expensive list view to list in arrow execution (#6304) @asubiotto
• chore: list contains over scalars (#6242) @blaginin
• perf[arrays]: move over all filter kernels to exec/reduce rules (#6298) @joseph-isaacs
• performance: Use push_unchecked when building indices for element take in fixedsizedlist take (#6222) @robert3005
• perf[vortex-array]: don't materialize const arrays in to_arrow_dictionary (#6265) @asubiotto
• Check read-only cache first (#6249) @gatesn
• Perf: Stream conversion from Parquet to Vortex to prevent OOM (#6228) @connortsui20
• Perf: Use optimized filter implementations is FilterArray::execute (#6189) @connortsui20
• coalesced request align all segments (#6158) @onursatici
• perf: CUDA async to host copy (#6085) @0ax1
• feat: make host to device copy async (#6075) @0ax1
• Remove duplicate ValidityVTable logic (#5975) @gatesn
• performance[vortex-array]: avoid is_valid call in varbinview builder (#6033) @asubiotto
• Only check for ConstantVTable (#6002) @gatesn
• do not validate is_sorted on runend (#6025) @onursatici
• perf: CUDA FoR loop unrolling (#6017) @0ax1
• feat: optimize repeatedScan for limit == 0 (#6015) @caicancai
• Remove unnecessary work from ListArray::validate (#6004) @AdamGS
• scan: make LazyScanStream initialization non-blocking (#5906) @godnight10061
• perf[sequence]: array constant compare execution kernel (#5795) @joseph-isaacs
• perf: reduce time_elapsed_opening overhead (#5879) @godnight10061

🐛 Bug Fixes

89 changes

• Measure output bytes in cuda bitunpack benchmarks (#6395) @robert3005
• fix[array]: decimal fill_null larger than buffer (#6388) @joseph-isaacs
• fix[array]: dict value nullable values (#6387) @joseph-isaacs
• fix[fuzz]: coverage report (#6375) @joseph-isaacs
• join ops before in format (#6370) @blaginin
• Include config.js in benchmark website docker image (#6379) @palaska
• Exporting struct array to arrow respects vortex nullabilty (#6365) @robert3005
• Fix null take on structs (#6364) @blaginin
• varbinview on device build skips validation (#6358) @onursatici
• fix: vortex-layout doc tests require zstd (#6179) @danking
• DF: Only calculate the size of the metadata cache entry once (#6342) @AdamGS
• bump time version to clear CI (#6335) @onursatici
• All-invalid compressor (#6330) @gatesn
• fix: Empty select (#6314) @gatesn
• Avoid some potential stack overflow cases with deep expression trees (#6321) @AdamGS
• shared array in dict layout (#6305) @onursatici
• skip sparse array validation for device resident arrays (#6315) @onursatici
• varbin to be constructed from buffer handle directly (#6317) @onursatici
• Do not use display_tree in error message (#6251) @gatesn
• Minor memory-related improvements in vortex-array and vortex-duckdb (#6299) @AdamGS
• use is_host to recursively check residency to skip validation (#6297) @onursatici
• Add slice kernel for BitPackedArray execute_parent (#6283) @onursatici
• select preserve validity of fields (#6260) @connortsui20
• fsst array: AllTrue and AllFalse are handled by filter fn (#6272) @blaginin
• fix: separate ptx build profiles (#6252) @0ax1
• fix[cuda]: Fix gpu_scan test (#6208) @a10y
• [benchmark-website] Fix mobile charts, better colors, better tooltip (#6226) @palaska
• fix: exclude flaky benchmarks from codspeed (#6220) @0ax1
• fix: skip buffer validation if on device (#6219) @0ax1
• fix: simd build (#6217) @0ax1
• chore[cuda]: fix windows (#6206) @joseph-isaacs
• Fix: dont do lazy filters on write (#6194) @connortsui20
• fix: evaluate OR for validity (#6184) @blaginin
• fix: cuda ptx folder (#6170) @joseph-isaacs
• Fix: Comment on a failed benchmark (#6181) @connortsui20
• fix[alp]: cast didn't cast patches (#6174) @joseph-isaacs
• fix[fuzz]: fix gpu fuzzer cfg flags (#6165) @joseph-isaacs
• fix[gpu]: synchronous copy to host (#6157) @0ax1
• fix[fuzz]: single sesssion for the fuzzer (#6155) @joseph-isaacs
• fix: sample multiples of 1024 (#6151) @danking
• fix: only link nvcomp for linux (#6144) @0ax1
• fix: fastlanes canonicalize benchmarks (#6141) @0ax1
• fix[fsst]: execute_parent empty validity non empty filter (#6133) @joseph-isaacs
• fix[vortex-array]: fix filter of sliced list (#6113) @asubiotto
• fix[list]: filter list execute parent (#6119) @blaginin
• fix[dtype]: fix typo in conversion from dtype to decimaltype (#6121) @joseph-isaacs
• fix[bitpacking]: filter (#6120) @joseph-isaacs
• fix[vortex-...

0.58.0

Changes

✨ Features

• Feature: Make vortex-tui a library (#5779) @connortsui20

🚀 Performance

• End DataFusion stream early if file can be pruned away by dynamic expr (#5874) @AdamGS

🧰 Maintenance

• chore: add overflow tests for unsigned types in BigCast (#5866) @caicancai

0.57.2

Changes

🐛 Bug Fixes

• fix: Move from macos-13 runners to macos-15-intel runners (#5875) @robert3005

0.57.1

Changes

🐛 Bug Fixes

• fix: Correctly pin maturin to 1.10.2 (#5873) @robert3005

0.57.0

Changes

⚠️ Breaks

• Fix BoolArray::new to take BitBuffer (#5690) @gatesn
• Lazy array expression evaluation (#5625) @gatesn
• Do not inline vortex_array into vortex:: namespace (#5584) @gatesn
• Move stats into expr (#5582) @gatesn
• Combine Array VTable with Encoding (#5462) @gatesn
• Consolidate into stat_expression function (#5366) @gatesn
• chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs

✨ Features

• feat: teach TemporalMetadata about its relationship to DTypes (#5864) @danking
• Pluggable expression conversion for DF (#5853) @AdamGS
• Introduce extendable access plan for DF scans (#5855) @AdamGS
• Upgrade DF to 51 (#5189) @AdamGS
• Benchmarks re-org (#5676) @AdamGS
• Push down cast (#5803) @gatesn
• ListView to List (#5802) @gatesn
• feat[array]: stats + validity in display array (#5799) @joseph-isaacs
• deterministic encoding ordering (#5770) @onursatici
• feat[fixed-size-list]: impl repeat and slice (#5751) @joseph-isaacs
• feat[compute]: compare decimal and varbin (#5757) @joseph-isaacs
• feat[datafusion]: Support DataFusion PartitionedFile extensions for external index (#5556) @haohuaijin
• FSST Filter Kernel (#5755) @gatesn
• perf[ci]: stopped instances (#5753) @joseph-isaacs
• Feature: makes the decompressor and compressor methods public (#5752) @connortsui20
• Feature: add fused bitpack filter kernel (#5741) @connortsui20
• Execute parent (#5736) @gatesn
• feat[array]: duckdb executor (#5737) @joseph-isaacs
• Arrow Executor (#5647) @gatesn
• Feature: add primitive casting (#5718) @connortsui20
• Feature: add some Cast infrastructure (#5708) @connortsui20
• feat[array]: dynamic expr execute (#5714) @joseph-isaacs
• Feature: add append_array_as_list methods to the list arrays (#5707) @connortsui20
• Feature: implement DictArray batched execute (#5686) @connortsui20
• feat[array]: list contains scalar (#5713) @joseph-isaacs
• feat[vector]: Eq (#5681) @joseph-isaacs
• Feature: add zero and null constructors for scalars (#5705) @connortsui20
• Feature: batch execute for DeltaArray (#5667) @connortsui20
• feat[vector]: slice varbinview (#5697) @joseph-isaacs
• Feature: add take for the view vectors (#5682) @connortsui20
• Feature: Implement take on more vectors + stubs (#5653) @connortsui20
• feature: add __version__ attribute to Python package (#5641) @d33bs
• Feature: add take for primitive and bool vectors (#5540) @connortsui20
• vx layout display (#5650) @joseph-isaacs
• Feature: added properties to VortexScanner (#5500) @sherlockbeard
• Add --version arg to vx (#5646) @palaska
• Feature: add total read size counter (#5600) @harryscholes
• feat[scalar_fn]: binary ops (#5617) @joseph-isaacs
• Execute ScalarFn arrays (#5611) @gatesn
• FilterArray (#5610) @gatesn
• Add GetItemFn (#5609) @gatesn
• Add MaskFn (#5606) @gatesn
• feat[scalar_fn]: is_null (#5601) @joseph-isaacs
• Scalar functions (#5561) @gatesn
• feat[buffer]: use BufferHandle in more places (#5567) @joseph-isaacs
• feat: alp decompress to vector (#5548) @0ax1
• feat[gpu]: use BufferHandle to the array vtable build (#5565) @joseph-isaacs
• feat[vortex-expr]: is_fallible analysis (#5546) @joseph-isaacs
• Feature: add take compute for slices (#5537) @connortsui20
• feat[vortex-expr]: is_fallible expression analysis (#5531) @joseph-isaacs
• feat[vortex-expr]: add a is_null_sensitive expression analysis (#5528) @joseph-isaacs
• Feature: add a take compute trait (#5503) @connortsui20
• Feature: add impl owned filter impls (#5493) @connortsui20
• Feature: add filter impl for FixedSizeListVector (#5494) @connortsui20
• Vector cast (#5461) @gatesn
• Scalar arithmetic (#5460) @gatesn
• feat[cli]: add scroll for tree encoding view (#5497) @a10y
• Revert "Revert "chore[ci]: add windows test to CI (#5439)"" (#5486) @joseph-isaacs
• More vector scalars (#5459) @gatesn
• feature: Support std_file on windows as it's used by cxx bindings (#5450) @robert3005
• Feature: implement bitpacking batch execute (#5446) @connortsui20
• feat[vortex-array]: added debug to expr/array reduce rules (#5442) @joseph-isaacs
• feat: publish vortex-spark-all assembly JAR (#5435) @a10y
• feat[vortex-array] add an array session rule registry (#5414) @joseph-isaacs
• Feature: add spare_capacity_mut to PVectorMut (#5287) @connortsui20
• feat: pipeline-pcodec (#5415) @0ax1
• Add VectorOps::clear (#5409) @gatesn
• feat[vortex-array]: expr array that represents lazy computation (#5400) @joseph-isaacs
• feat[vortex-expr]: reduce_parent has typed parent (#5398) @joseph-isaacs
• feat: cast expression (#5389) @danking
• feat: PyExpr::evaluate (#5388) @danking
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs
• Faster Pipelines (#5352) @gatesn
• feat[vortex-expr]: better debug print (#5374) @joseph-isaacs

🚀 Performance

• perf: list contains scalar scalar (#5792) @0ax1
• Upgrade DF to 51 (#5189) @AdamGS
• performance[vortex-array]: don't call is_valid to count bytes in varbinview (#5814) @asubiotto
• perf[array]: extension pushdown filter + constant (#5794) @joseph-isaacs
• perf[expr]: between not fallible (#5798) @joseph-isaacs
• perf[dtp]: push down compare over dtp constant second subsecond (#5793) @joseph-isaacs
• perf[date-time-parts]: compressor handle constant and pushdown rules (#5778) @joseph-isaacs
• perf[duckdb]: dont push down pack & dedup filters for ddb (#5773) @joseph-isaacs
• ci: use windows-x64 pool for Windows tests (#5756) @joseph-isaacs
• feat[duckdb]: optimize array (#5738) @joseph-isaacs
• Perf: optimize take_scalar (#5723) @connortsui20
• Make sure all encodings are enabled for DF (#5720) @AdamGS
• Perf: Optimized bool take (#5701) @connortsui20
• Run optimizer on root array only on (#5710) @gatesn
• perf[array]: Constant kernel filter pushdown (#5698) @joseph-isaacs
• perf[array]: with_children vtable function (#5694) @joseph-isaacs
• perf[ci]: speed up ci MORE (#5576) @joseph-isaacs
• perf(ci): skip rust-cache on runs-on runners with S3 sccache (#5570) @joseph-isaacs
• chunked array zip (#5530) @onursatici
• varbinview zip kernel (#5526) @onursatici
• chore[vortex-dict]: numeric pushdown & clean up the dict ctor (#5438) @joseph-isaacs
• Handwritten pipeline benchmark and analysis (#5408) @connortsui20
• Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
• perf[dict]: all_values_referenced to allow pushdown optimisation. (#5416) @joseph-isaacs
• perf[vortex-expr]: replace can stop looking at subtree if it finds a match (#5405) @joseph-isaacs
• actually make ZCTL (#5402) @a10y
• fix: performance regression for filtering ListView (#5390) @a10y
• fix[vortex-dict]: faster min_max for dict array (#5288) @joseph-isaacs
• fix: yo dawg, I heard you like inlining (#5363) @a10y

🐛 Bug Fixes

81 changes

• fix: enable all features for vortex-mask docsrs and enable serde-derive when serde is enabled (#5871) @robert3005
• fix[array]: correct the handling of to_arrow execution and struct casting (#5867) @joseph-isaacs
• fix[ci]: pin maturin due to fix upstream error (#5868) @joseph-isaacs
• fix: int compressor mul overflow bp (#5854) @joseph-isaacs
• Fix: IPC AsyncMessageReader does not handle partial reads (#5758) @connortsui20
• Fix: incorrect child vector size in duckdb list array import (#5740) @connortsui20
• Fix: bring back lance compression benchmark (#5840) @connortsui20
• Fix: don't fail on missing lance for datafusion (#5841) @connortsui20
• Fix: CI sql script (#5838) @connortsui20
• Fix: CI SQL Remote (#5836) @connortsui20
• Fix: finally fix CI (#5835) @connortsui20
• Fix: actually actually fix CI (#5830) @connortsui20
• Fix: benchmark name for bench.yml (#5828) @connortsui20
• Fix: remove lance feature from vortex-bench CI (#5823) @connortsui20
• Fix: decimal cast error message (#5821) @connortsui20
• Fix Arrow list export (#5804) @gatesn
• Fix: support widening of ListViewVector offsets and sizes (#5796) @connortsui20
• perf[layout]: apply then filter (#5788) @joseph-isaacs
• perf[date-time-parts]: compressor handle constant and pushdown rules p2 (#5786) @joseph-isaacs
• fix: disable noisy btrblocks micro-benchmarks on codspeed (#5782) @0ax1
• Fix arrow conversion (#5769) @gatesn
• fix: downcast in list new_vector_exporter (#5768) @0ax1
• fix[duckdb]: no to_struct in exporter (#5750) @joseph-isaacs
• Fix: remove memory limit on fuzzers (#5763) @connortsui20
• Fix: ListArray take incorrect validity (#5762) @connortsui20
• fix: list contains (#5748) @0ax1
• fix[layout]: soundness note on values eval (#5747) @joseph-isaacs
• fix: only validate non-null decimals (#5745) @0ax1
• fix: zstd frame step width for alignment (#5730) @0ax1
• fix[duckdb]: remove validity mask from vector export (#5739) @joseph-isaacs
• fix: Buffer slice end does not need to be aligned (#5734) @0ax1
• Fix Arrow struct export (#5731) @gatesn
• Disable GHArchive benchmarks to unblock perf reference in CI (#5729) @AdamGS
• feat[array]: fixup operator stuff (#5711) @joseph-isaacs
• Reduce optimizer logging to debug (#5702) @gatesn
• fix: list take offset type (#5679) @0ax1
• fix: make error type explicit (#5696) @0ax1
• Avoid taking write lock on VortexSession::get (#5688) @gatesn
• Fix: claude action concurrency group (#5680) @connortsui20
• Temp fix: Make take bound by Copy instead of NativePType (#5666) @connortsui20
• Fix: add RUSTFLAGS="--cfg vortex_nightly" to all nightly tests (#5655) @connortsui20
• Fix: portable SIMD take implementation (#5654) @connortsui20
• Fix session get-or-default (#5662) @gatesn
• Include zstd encoding in IO fuzzer (#5661) @AdamGS
• Feature: added properties to VortexScanner (#5500) @sherlockbeard
• fix[vortex-array]: fix take on varbinviews with NULL indices...

0.57.0-rc.0

Changes

⚠️ Breaks

• Do not inline vortex_array into vortex:: namespace (#5584) @gatesn
• Move stats into expr (#5582) @gatesn
• Combine Array VTable with Encoding (#5462) @gatesn
• Consolidate into stat_expression function (#5366) @gatesn
• chore[vortex-array]: remove SerdeVTable (#5387) @joseph-isaacs
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs

✨ Features

• feat[scalar_fn]: binary ops (#5617) @joseph-isaacs
• Execute ScalarFn arrays (#5611) @gatesn
• FilterArray (#5610) @gatesn
• Add GetItemFn (#5609) @gatesn
• Add MaskFn (#5606) @gatesn
• feat[scalar_fn]: is_null (#5601) @joseph-isaacs
• Scalar functions (#5561) @gatesn
• feat[buffer]: use BufferHandle in more places (#5567) @joseph-isaacs
• feat: alp decompress to vector (#5548) @0ax1
• feat[gpu]: use BufferHandle to the array vtable build (#5565) @joseph-isaacs
• feat[vortex-expr]: is_fallible analysis (#5546) @joseph-isaacs
• Feature: add take compute for slices (#5537) @connortsui20
• feat[vortex-expr]: is_fallible expression analysis (#5531) @joseph-isaacs
• feat[vortex-expr]: add a is_null_sensitive expression analysis (#5528) @joseph-isaacs
• Feature: add a take compute trait (#5503) @connortsui20
• Feature: add impl owned filter impls (#5493) @connortsui20
• Feature: add filter impl for FixedSizeListVector (#5494) @connortsui20
• Vector cast (#5461) @gatesn
• Scalar arithmetic (#5460) @gatesn
• feat[cli]: add scroll for tree encoding view (#5497) @a10y
• Revert "Revert "chore[ci]: add windows test to CI (#5439)"" (#5486) @joseph-isaacs
• More vector scalars (#5459) @gatesn
• feature: Support std_file on windows as it's used by cxx bindings (#5450) @robert3005
• Feature: implement bitpacking batch execute (#5446) @connortsui20
• feat[vortex-array]: added debug to expr/array reduce rules (#5442) @joseph-isaacs
• feat: publish vortex-spark-all assembly JAR (#5435) @a10y
• feat[vortex-array] add an array session rule registry (#5414) @joseph-isaacs
• Feature: add spare_capacity_mut to PVectorMut (#5287) @connortsui20
• feat: pipeline-pcodec (#5415) @0ax1
• Add VectorOps::clear (#5409) @gatesn
• feat[vortex-array]: expr array that represents lazy computation (#5400) @joseph-isaacs
• feat[vortex-expr]: reduce_parent has typed parent (#5398) @joseph-isaacs
• feat: cast expression (#5389) @danking
• feat: PyExpr::evaluate (#5388) @danking
• feat[vortex-expr]: add expr rewrite rules (#5348) @joseph-isaacs
• Faster Pipelines (#5352) @gatesn
• feat[vortex-expr]: better debug print (#5374) @joseph-isaacs

🚀 Performance

• perf[ci]: speed up ci MORE (#5576) @joseph-isaacs
• perf(ci): skip rust-cache on runs-on runners with S3 sccache (#5570) @joseph-isaacs
• chunked array zip (#5530) @onursatici
• varbinview zip kernel (#5526) @onursatici
• chore[vortex-dict]: numeric pushdown & clean up the dict ctor (#5438) @joseph-isaacs
• Handwritten pipeline benchmark and analysis (#5408) @connortsui20
• Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
• perf[dict]: all_values_referenced to allow pushdown optimisation. (#5416) @joseph-isaacs
• perf[vortex-expr]: replace can stop looking at subtree if it finds a match (#5405) @joseph-isaacs
• actually make ZCTL (#5402) @a10y
• fix: performance regression for filtering ListView (#5390) @a10y
• fix[vortex-dict]: faster min_max for dict array (#5288) @joseph-isaacs
• fix: yo dawg, I heard you like inlining (#5363) @a10y

🐛 Bug Fixes

35 changes

• fix: avoid ListBuilder (#5636) @a10y
• fix: disable flaky bench for codspeed (#5580) @0ax1
• fix: don't bench slice_arrow_buffer on codspeed (#5578) @0ax1
• fix: fixed seed for micro-benchmarks (#5574) @0ax1
• fix: don't clone in micro-benchmarks (#5573) @0ax1
• fix: Decimal sum doesn't panic but returns null on overflow (#5564) @robert3005
• fix[dict]: min_max must not be null (#5560) @joseph-isaacs
• fix[display]: escape strings (#5551) @joseph-isaacs
• fix: benchmark black box handling (#5558) @0ax1
• fix: make micro-benchmark labels stable (#5557) @0ax1
• Fix: Schema projection order mismatch from table/file schema (#5541) @brancz
• fix: remove print (#5547) @joseph-isaacs
• fix: assert_arrays_eq macro reexports format function to hide itertools dep (#5543) @robert3005
• fix[mask]: intersect_by_rank and bit to arrow (#5545) @joseph-isaacs
• fix: Restore fast constant sum behaviour, don't fuzz floating point sum compute function (#5527) @robert3005
• Fix: ConstantArray float sum behaves the same way as primitive array (#5513) @robert3005
• Fix: this may have been a merge conflict (#5510) @connortsui20
• fix[dict]: not (filter/take) set_all_values_referenced (#5496) @joseph-isaacs
• fix: Sum from stats has to account for accumulator (#5445) @robert3005
• fix: benchmark input handling (#5488) @0ax1
• fix: handling of struct fields in Spark (#5453) @a10y
• Revert "chore[ci]: add windows test to CI (#5439)" (#5449) @0ax1
• Fix: remove unreachable cases in bitpack patches (#5444) @connortsui20
• fix: FFI returns a string/binary instead of requiring the caller to guess the necessary memory (#5433) @robert3005
• Fix: Correctly handle inconsistent file and table schema order in vortex-datafusion (#5436) @brancz
• Fix: incorrect offset for nulls + don't chunk elements (#5423) @connortsui20
• fix[ci]: use run_id in the s3 benchmark runs (#5421) @joseph-isaacs
• fix: add missing dependency: typing-extensions (#5420) @danking
• fix: vx_struct_fields_field_dtype takes usize instead of u64 (#5396) @robert3005
• fix: BitView slice filter (#5381) @a10y
• fix: cast must cast its stats too (#5386) @danking
• fix: VarBin overflow on take (#5361) @a10y
• fix[vortex-array]: correct convert to arrow from vortex (#5378) @joseph-isaacs
• Use the specified concurrency value when reading from a ObjectStoreIoSource (#5371) @nrc
• fix: yo dawg, I heard you like inlining (#5363) @a10y

📖 Documentation

• docs: fix typo in readme (#5627) @0ax1
• Chore: allow searching subcrates in vortex rustdoc (#5612) @connortsui20
• Describe upcoming API breaks in docs (#5583) @gatesn
• TSC Agenda 2025-11-25 (#5523) @gatesn
• Docs: document chunk_offsets in Patches (#5429) @connortsui20

🧰 Maintenance

87 changes

• Add advisory semver checks to CI (#5645) @gatesn
• fix(deps): update protobuf monorepo to v4.33.2 (#5644) @renovate[bot]
• chore(deps): update all patch updates (#5643) @renovate[bot]
• fix platform/arch for runs-on config (#5622) @a10y
• ci: add runs-on.yml that points to prebuilt AMI (#5620) @a10y
• ci: add cache-suffix to isolate sanitizer build cache (#5604) @joseph-isaacs
• feat[fuzz]: add WASM fuzzer support for wasmfuzz (#5575) @joseph-isaacs
• chore(deps): update crate-ci/typos action to v1.40.0 (#5596) @renovate[bot]
• chore(deps): update tokio-tracing monorepo (#5595) @renovate[bot]
• chore(deps): update all patch updates (#5593) @renovate[bot]
• chore: inline alp decompress_from_buffer fn (#5587) @0ax1
• Remove Operator VTable (#5588) @gatesn
• Test implementation of cast using ScalarFn (#5586) @gatesn
• Remove ExprArray and ExprOptimizer (#5585) @gatesn
• Move stats into expr (#5582) @gatesn
• chore[ci]: bump codspeed runner (#5579) @joseph-isaacs
• chore: turn byte factory macro into function (#5577) @0ax1
• ci: simplify codspeed sharding (#5572) @0ax1
• chore: speed up taplo format (#5569) @robert3005
• chore: Remove codspeed warning about deprecated mode (#5562) @robert3005
• Chore: use item imports in bench-vortex (#5554) @connortsui20
• Remove clone from array vtable (#5552) @gatesn
• Chore: check taplo formatting in pre-commit (#5549) @connortsui20
• chore[*]: prefer #[expect(..)] over #[allow(..)] p2 (#5533) @joseph-isaacs
• chore[*]: prefer #[expect(..)] over #[allow(..)] (#5498) @joseph-isaacs
• chore: benchmark hygiene (#5499) @0ax1
• chore[ffi]: remove header gen warning not nightly (#5529) @joseph-isaacs
• varbinview zip bench (#5518) @onursatici
• chore: apply imports_granularity = "Item" everywhere expect bench-vortex (#5521) @joseph-isaacs
• chore: fix error message for unwrap_scalar (#5515) @xumingming
• Chore: remove unstable pipeline benchmark (#5512) @connortsui20
• Expression execution to be passed input vectors (#5507) @gatesn
• Chore: Use full item paths for every import is vortex-vector (#5509) @connortsui20
• Chore: Fully qualify imported items (#5504) @connortsui20
• Chore: Even more stable benchmarks (#5506) @connortsui20
• Chore: make pipeline benchmarks less susceptible to noise (#5502) @connortsui20
• Chore: refactor dict module (#5501) @connortsui20
• chore: minimize with_inputs work for benchmarks (#5492) @0ax1
• Preapproved issue template (#5489) @gatesn
• chore[dbp]: remove useless_conversion (#5487) @joseph-isaacs
• Update Rust crate pyo3 to 0.27.0 (#5478) @renovate[bot]
• Update Rust crate goldenfile to v1.9.1 (#5474) @renovate[bot]
• Update Rust crate tokio to v1.48.0 (#5480) @renovate[bot]
• Update Rust crate insta to v1.44.1 (#5475) @renovate[bot]
• Update all patch updates (#5470) @renovate[bot]
• Update Rust crate noodles-bgzf to 0.44.0 (#5476) @renovate[bot]
• Update protobuf monorepo to v4.33.1 (#5472) @renovate[bot]
• Update Rust crate cudarc to 0.18.0 (#5473) @renovate[bot]
• Update plugin com.diffplug.spotless to v8.1.0 (#5471) @[renovate[bot]](http...

🌪 0.56.0

Changes

⚠️ Breaks

• chore[array/expr]: move vortex-expr into vortex_array::expr (#5269) @joseph-isaacs

✨ Features

• Move BitView to vortex-buffer and make generic over N (#5353) @gatesn
• feat: bump duckdb to 1.4.2 (#5337) @0ax1
• arrow -> vortex for dict arrays (#5316) @onursatici
• feat: automated fuzzing issue creation with Claude (#5292) @joseph-isaacs
• Feature: add aligned bitpacked pipeline kernel (#5299) @connortsui20
• feat: impl expand for Buffer & BufferMut (#5270) @0ax1
• feat: use Sparse compression for floats, strings (#5237) @a10y

🚀 Performance

• perf: BitBuffer::from(Vec) use collect_bool (#5325) @joseph-isaacs
• perf: optimize expand (#5285) @0ax1
• fix[vortex-dict]: avoid full materialization in min_max (#5268) @asubiotto

🐛 Bug Fixes

13 changes

• fix: add permissions to fuzzer fix automation workflow (#5345) @joseph-isaacs
• fix: SequenceVTable bounds check (#5341) @blaginin
• fix: unblock the fuzzers (#5335) @a10y
• Fix: handle validity in bitpack pipeline kernel (#5330) @connortsui20
• Fix zero-copy pipeline input (#5324) @gatesn
• fix: update Claude Code Action to v1 parameter format (#5314) @joseph-isaacs
• fix vortex-gpu compilation (#5315) @onursatici
• fix[vortex-datafusion]: check field exists in get_field pushdown (#5295) @asubiotto
• fix[vortex-dict]: avoid full materialization in min_max (#5268) @asubiotto
• fix[array]: sum with initial value to fix op assoc (#5278) @joseph-isaacs
• fix: vortex-tui browse with search (#5282) @a10y
• handle null views correctly on varbinview compaction (#5279) @onursatici
• fix: compare for struct with constant items (#5271) @a10y

🧰 Maintenance

34 changes

• chore: All of vortex-gpu requires compiler flag (#5362) @robert3005
• chore(deps): update rust crate bytes to v1.11.0 (#5358) @renovate[bot]
• chore(deps): update all patch updates (#5357) @renovate[bot]
• chore: bug is not a sufficient label (#5351) @danking
• chore: document Expr.getitem (get a field of a struct array) (#5344) @danking
• fix: cfg disable unix-specific operations (#5326) @danking
• chore: duckdb e2e roundtrip tests (#5342) @0ax1
• feat: add manual trigger support to fuzzer fix automation (#5323) @joseph-isaacs
• chore: bump duckdb version to 1.4.2 in gh actions (#5339) @0ax1
• test: more iter_bits test cases (#5336) @0ax1
• chore: add automated crash reporting for array_ops fuzzing (#5317) @joseph-isaacs
• chore: implement Filter for new MaskIndices type for all vectors (#5333) @a10y
• Chore: add todo and clean a bit in pipeline step (#5334) @connortsui20
• chore[claude]: update claude.md (#5318) @joseph-isaacs
• feat: Add Stage 2 automated fix workflow for fuzzer crashes (#5319) @joseph-isaacs
• chore: bump claude-code-action to v1.0.16 (#5312) @joseph-isaacs
• chore(deps): update gradle to v9 (#4085) @renovate[bot]
• chore(deps): update plugin com.palantir.git-version to v4.2.0 (#5309) @renovate[bot]
• chore(deps): update plugin com.vanniktech.maven.publish to v0.35.0 (#5310) @renovate[bot]
• chore(deps): update plugin com.palantir.consistent-versions to v3.7.0 (#5308) @renovate[bot]
• fix(deps): update dependency ch.qos.logback:logback-classic to v1.5.21 (#5307) @renovate[bot]
• chore(deps): update actions/download-artifact action to v6 (#5305) @renovate[bot]
• chore(deps): update actions/upload-artifact action to v5 (#5306) @renovate[bot]
• Chore: move filter files around (#5300) @connortsui20
• Unblock release by publishing GPU crates (#5293) @AdamGS
• chore: make ArrayAccessor infallible (#5303) @a10y
• move vortex-dict into vortex-array (#5289) @onursatici
• Pipeline Calling Convention 2 (#5291) @gatesn
• Clarify calling convention for pipeline kernels (#5286) @gatesn
• API and partial implementation of sparse pipelined exection (#5284) @gatesn
• chore: add slack to readme (#5281) @lwwmanning
• document reproducible writes (#5280) @onursatici
• chore[array/expr]: move vortex-expr into vortex_array::expr (#5269) @joseph-isaacs
• chore: serialize for Mask (#5267) @blaginin