What
QDP documents .feather as a supported CUDA file format. PyArrow writes Feather V2 files with LZ4 compression by default, but the QDP Arrow dependency is built without IPC compression support. Files produced by the standard PyArrow API therefore fail in both direct encoding and the Rust-backed loader.
Steps to reproduce:
from pathlib import Path
import numpy as np
import pyarrow as pa
import pyarrow.feather as feather
from qumat_qdp import QdpEngine, QuantumDataLoader
path = Path("vectors.feather")
values = np.eye(4, dtype=np.float64)
vectors = pa.FixedSizeListArray.from_arrays(
pa.array(values.reshape(-1)),
list_size=4,
)
feather.write_feather(pa.table({"data": vectors}), path)
QdpEngine().encode(path, num_qubits=2, encoding_method="amplitude")
list(
QuantumDataLoader()
.backend("rust")
.qubits(2)
.batches(1, size=2)
.source_file(str(path))
)
Both paths fail with an error containing:
Invalid argument error: lz4 IPC decompression requires the lz4 feature
Actual result: Both documented CUDA file APIs reject a Feather V2 file written with PyArrow's default compression.
Expected result: QDP reads Feather files written with PyArrow's default settings through direct encoding and the Rust-backed loader.
This issue is limited to compressed Arrow IPC/Feather input. PyTorch .pt and .pth input is tracked separately because it fails through a different reader path.
Environment
- OS: Linux
- Python version: 3.12.12
- Qumat version: 0.7.0.dev0 (
qumat-qdp 0.3.0.dev0)
- PyArrow version: 25.0.1
- GPU: NVIDIA GeForce RTX 4090
What
QDP documents
.featheras a supported CUDA file format. PyArrow writes Feather V2 files with LZ4 compression by default, but the QDP Arrow dependency is built without IPC compression support. Files produced by the standard PyArrow API therefore fail in both direct encoding and the Rust-backed loader.Steps to reproduce:
Both paths fail with an error containing:
Actual result: Both documented CUDA file APIs reject a Feather V2 file written with PyArrow's default compression.
Expected result: QDP reads Feather files written with PyArrow's default settings through direct encoding and the Rust-backed loader.
This issue is limited to compressed Arrow IPC/Feather input. PyTorch
.ptand.pthinput is tracked separately because it fails through a different reader path.Environment
qumat-qdp0.3.0.dev0)