Skip to content
Open
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
0345a1c
Add dimension separator as a type parameter
mkitti Mar 11, 2025
61786e3
Fix ZipStore constructor
mkitti Mar 12, 2025
cbb23ce
Fix ConsolidatedStore
mkitti Mar 12, 2025
e4630a9
Fix S3Store constructor
mkitti Mar 12, 2025
b9e175f
Add version as a type parameter
mkitti Mar 13, 2025
3624376
Check metadata for dimension_separator and zarr_format
mkitti Mar 18, 2025
2b3bbb2
Implement VersionStorage wrapper rather than modifying AbstractStorage
mkitti Mar 26, 2025
5f35ebf
Fix ConslidatedStore wrapper around HTTP
mkitti Mar 31, 2025
c685387
Add getproperty forwarding from VersionedStorage
mkitti Mar 31, 2025
8d5606d
Add some tests for propertynames
mkitti Mar 31, 2025
a6fcc2b
Add Storage/versionstore.jl
mkitti Mar 31, 2025
f6883f8
Add VersionedStorage param change constructors
mkitti Apr 1, 2025
3cf746d
Add V2 chunk encoding support
mkitti May 6, 2025
d218dc2
Fix Base.UInt8 constructor for ASCIIChar
mkitti May 7, 2025
6f722b5
Add ZstdCompressor
nhz2 Mar 7, 2025
865dac7
fix typo
nhz2 Mar 7, 2025
6d7dc21
Prototype Zarr v3 support
mkitti Apr 1, 2025
b394457
Modify tutorial to match current storage display
mkitti Apr 1, 2025
8e71a33
Ensure configuration key exists
mkitti May 8, 2025
08288fd
Change VersionedStore to FormattedStore
mkitti May 19, 2025
5bb7358
Merge pull request #1 from mkitti/mkitti-formatted-store
mkitti May 23, 2025
020b3dd
Merge branch 'mkitti-dimension-separator-type-parameter' into mkitti-…
mkitti May 30, 2025
0046e14
Add {get,write}attrs for FormattedStore{3}
mkitti Jun 2, 2025
34afb27
Add separator function for V2ChunkKeyEncoding
mkitti Jun 2, 2025
514ba87
Fix formattedstore, add writemetadata
mkitti Jun 2, 2025
4ce5895
Attempt to allow for Zarr v3 array creation
mkitti Jun 2, 2025
3298a5c
Fix Zarr v3 array creation
mkitti Jun 2, 2025
646ba9c
Implement CRC32c Zarr v3 codec
mkitti Jun 2, 2025
d4217fb
Merge branch 'master' into mkitti-v3-prototype
mkitti Jun 4, 2025
07352f3
Fix spelling of Evaluate in comment
mkitti Jun 4, 2025
42b2519
Fix default chunk_key_encoding
mkitti Aug 27, 2025
9722a1a
Merge branch 'master' into mkitti-v3-prototype
mkitti Oct 12, 2025
32da023
dont
lazarusA Nov 16, 2025
1df9efe
adds type AbstractMetadata
lazarusA Nov 16, 2025
db8a08c
dispatch
lazarusA Nov 16, 2025
136470e
fix tests
lazarusA Nov 16, 2025
616f563
py v3 baseline
lazarusA Nov 16, 2025
df7cbf4
julia version
lazarusA Nov 16, 2025
f369106
claude's sharding version, debug, integrate now
lazarusA Nov 17, 2025
c3ba31e
offset nbytes order
lazarusA Nov 17, 2025
7913fbc
Merge branch 'master' into continue_v3_prototype
mkitti Nov 24, 2025
b741d64
Merge branch 'master' into continue_v3_prototype
mkitti Nov 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ Manifest.toml
docs/build
*.zarr
.CondaPkg
.vscode
test/CondaPkg.toml
2 changes: 2 additions & 0 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ version = "0.9.5"
[deps]
AWSS3 = "1c724243-ef5b-51ab-93f4-b0a88ac62a95"
Blosc = "a74b3585-a348-5f62-a45c-50e91977d574"
CRC32c = "8bf52ea8-c179-5cab-976a-9e18b702a9bc"
ChunkCodecCore = "0b6fb165-00bc-4d37-ab8b-79f91016dbe1"
ChunkCodecLibZlib = "4c0bbee4-addc-4d73-81a0-b6caacae83c8"
ChunkCodecLibZstd = "55437552-ac27-4d47-9aa3-63184e8fd398"
Expand All @@ -27,6 +28,7 @@ Blosc = "0.5, 0.6, 0.7"
ChunkCodecCore = "1"
ChunkCodecLibZlib = "1"
ChunkCodecLibZstd = "1"
CRC32c = "1.10, 1.11"
DataStructures = "0.17, 0.18, 0.19"
DateTimes64 = "1"
DiskArrays = "0.4.2"
Expand Down
2 changes: 1 addition & 1 deletion docs/src/tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ Order : C
Read-Only : false
Compressor : Zarr.BloscCompressor(0, 3, "zstd", 1)
Filters : nothing
Store type : Dictionary Storage
Store type : Zarr.FormattedStore{2, '.', Zarr.DictStore}(Dictionary Storage)
No. bytes : 400000000
No. bytes stored : 2412289
Storage ratio : 165.81761140559857
Expand Down
49 changes: 49 additions & 0 deletions src/Codecs/Codecs.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
module Codecs

using JSON: JSON

"""
abstract type Codec

The abstract supertype for all Zarr codecs

## Interface

All subtypes of `Codec` SHALL implement the following methods:

- `zencode(a, c::Codec)`: compress the array `a` using the codec `c`.
- `zdecode(a, c::Codec, T)`: decode the array `a` using the codec `c`
and return an array of type `T`.
- `JSON.lower(c::Codec)`: return a JSON representation of the codec `c`, which
follows the Zarr specification for that codec.
- `getCodec(::Type{<:Codec}, d::Dict)`: return a codec object from a given
dictionary `d` which contains the codec's parameters according to the Zarr spec.

Subtypes of `Codec` MAY also implement the following methods:

- `zencode!(encoded, data, c::Codec)`: encode the array `data` using the
codec `c` and store the result in the array `encoded`.
- `zdecode!(data, encoded, c::Codec)`: decode the array `encoded`
using the codec `c` and store the result in the array `data`.

Finally, an entry MUST be added to the `VN.codectypes` dictionary for each codec type where N is the
Zarr format version.
This must also follow the Zarr specification's name for that compressor. The name of the compressor
is the key, and the value is the compressor type (e.g. `BloscCodec` or `NoCodec`).

For example, the Blosc codec is named "blosc" in the Zarr spec, so the entry for [`BloscCodec`](@ref)
must be added to `codectypes` as `codectypes["blosc"] = BloscCodec`.
"""

abstract type Codec end

zencode(a, c::Codec) = error("Unimplemented")
zencode!(encoded, data, c::Codec) = error("Unimplemented")
zdecode(a, c::Codec, T::Type) = error("Unimplemented")
zdecode!(data, encoded, c::Codec) = error("Unimplemented")
JSON.lower(c::Codec) = error("Unimplemented")
getCodec(::Type{<:Codec}, d::Dict) = error("Unimplemented")

include("V3/V3.jl")

end
Loading
Loading