Skip to content

Taking serialization seriously #1988

Open
@ToucheSir

Description

@ToucheSir

As a rule, we've been very careful about breaking changes to struct internals and making sure users have a migration path between versions. However, the serialization librar(ies) we recommend using with Flux throw all that out the window because they rely on exactly those struct internals. This leads to gnarly issues like #1984.

To that end, I'd like to brainstorm ideas of how we can reduce our dependence on serialization library magic. One idea would be to implement an interface like StructTypes or ArrowTypes. Downstream libraries like LightBSON or LegolasFlux can then make use of that interface for more controlled serialization. On our side, we'd also have more control over what and how properties are (de)serialized, including adding shims for discrepancies between package versions.

The main downside of relying on an interface package is lack of adoption. However, I don't think that's the end of the world since users can always fall back to a BSON.jl or JLD2. If we can lead by example and help accelerate widespread adoption, it would be quite nice to not have to constantly worry about changing struct fields!

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions