|
| 1 | +**This is a work-in-progress. Please consider starring or watching the repository |
| 2 | +to stay up to date.** |
| 3 | + |
| 4 | +*** |
| 5 | + |
1 | 6 | 
|
2 | 7 |
|
3 | 8 | JSON BinPack is an open-source binary JSON serialization format with a **strong
|
4 | 9 | focus on space efficiency**. It supports schema-driven and schema-less modes to
|
5 | 10 | encode any [JSON](https://www.json.org) document given a matching [JSON Schema
|
6 | 11 | 2020-12](http://json-schema.org) definition.
|
7 | 12 |
|
8 |
| -*** |
| 13 | + |
9 | 14 |
|
10 |
| -**This is a work-in-progress. Please consider starring or watching the repository |
11 |
| -to stay up to date.** |
| 15 | +- **Highly Space-efficient**: A reproducible benchmark study has proven JSON |
| 16 | + BinPack to be space-efficient in comparison to 12 alternative binary |
| 17 | + serialization formats in every considered case. Additionally, JSON BinPack |
| 18 | + typically provides higher average size reduction than general purpose |
| 19 | + compressors such as GZIP, LZMA and LZ4 with the highest compression levels. |
12 | 20 |
|
13 |
| -*** |
| 21 | + [Take a look at the benchmark](https://benchmark.sourcemeta.com) |
| 22 | + |
| 23 | +- **Based on JSON Schema**: JSON BinPack adopts the industry-standard schema |
| 24 | + language for JSON documents. Re-use the same schemas you already have in your |
| 25 | + OpenAPI, AsyncAPI, RAML or other specifications. No need to learn more |
| 26 | + domain-specific schema languages or attempt to translate between them. |
| 27 | + |
| 28 | + [Learn more about JSON Schema](https://json-schema.org/) |
| 29 | + |
| 30 | +- **Optional Schema**: JSON BinPack is a hybrid serialization format that runs |
| 31 | + in both schema-driven and schema-less modes. Thanks to JSON Schema, JSON |
| 32 | + BinPack allows you to be as loose or specific as your use case demands, even |
| 33 | + within the same document, without having to use different serialization |
| 34 | + technologies. |
| 35 | + |
| 36 | + [Read more about JSON Schema as a constraint system](https://modern-json-schema.com/json-schema-is-a-constraint-system) |
14 | 37 |
|
15 | 38 | Documentation
|
16 | 39 | -------------
|
17 | 40 |
|
18 |
| -Refer to the project website for documentation: |
| 41 | +Refer to the project website for reference documentation: |
19 | 42 | [https://jsonbinpack.sourcemeta.com](https://jsonbinpack.sourcemeta.com).
|
20 | 43 |
|
21 | 44 | Do you have any questions? Open a ticket on [GitHub
|
22 | 45 | Discussions](https://github.com/sourcemeta/jsonbinpack/discussions)!
|
| 46 | + |
| 47 | +Frequently Asked Questions |
| 48 | +-------------------------- |
| 49 | + |
| 50 | +### How does JSON BinPack compare to compression formats like GZIP? |
| 51 | + |
| 52 | +Our [reproducible benchmark study](https://benchmark.sourcemeta.com) shows that |
| 53 | +the semantic information provided by detailed schema definitions enables a |
| 54 | +serialization format such as JSON BinPack to often outperform general-purpose |
| 55 | +compressors configured with even their highest supported compression levels, |
| 56 | +specially for small and medium-sized JSON documents. For large JSON documents, |
| 57 | +you can experiment combining JSON BinPack with a compression format to |
| 58 | +potentially achieve greater space-efficiency. |
| 59 | + |
| 60 | +### JSON BinPack is space-efficient, but what about runtime-efficiency? |
| 61 | + |
| 62 | +When transmitting data over the Internet, time is the bottleneck, making |
| 63 | +computation much cheaper in comparison. Leaving extremely low-powered devices |
| 64 | +aside, trading more CPU cycles for better compression is a sensible choice. |
| 65 | +JSON BinPack particularly excels at improving network performance when |
| 66 | +transmitting data through unreliable and low-bandwidth connections on mobile |
| 67 | +and IoT systems, and it is still optimized to deliver the best runtime |
| 68 | +efficiency possible. |
| 69 | + |
| 70 | +If your use-case demands extreme runtime-efficiency or zero-copy serialization, |
| 71 | +such as in the case of intra-service or inter-process communication, we suggest |
| 72 | +you consider alternative serialization formats such as |
| 73 | +[FlatBuffers](https://google.github.io/flatbuffers/) and [Cap'n |
| 74 | +Proto](https://capnproto.org/). |
| 75 | + |
| 76 | +### How does JSON BinPack handle unknown fields? |
| 77 | + |
| 78 | +JSON BinPack serializes any instance that matches its JSON Schema definition. |
| 79 | +JSON Schema is an expressive constrain-based language that permits |
| 80 | +schema-writers to be as strict or loose on defining data as they wish. As a |
| 81 | +consequence of adopting JSON Schema, JSON BinPack elegantly supports encoding |
| 82 | +free-form data and unknown fields as long as the corresponding JSON Schema |
| 83 | +definition successfully validates the data. |
| 84 | + |
| 85 | +However, deserializing data that does not match the given JSON Schema |
| 86 | +definition is undefined behavior and will likely result in an exception being |
| 87 | +thrown. |
| 88 | + |
| 89 | +### How does JSON BinPack compare to [alternative]? |
| 90 | + |
| 91 | +We maintain a [live benchmark](https://benchmark.sourcemeta.com/) comparing |
| 92 | +JSON BinPack to various popular alternatives, including Protocol Buffers, CBOR, |
| 93 | +MessagePack, Apache Avro, and more. We also published an [academic |
| 94 | +paper](https://arxiv.org/abs/2211.12799) discussing the benchmark results in |
| 95 | +more detail. In summary, you can expect JSON BinPack to be as or more |
| 96 | +space-efficient than every alternative in every tested case. |
| 97 | + |
| 98 | +### How does JSON BinPack support schema evolution? |
| 99 | + |
| 100 | +In comparison to schema-driven alternatives like Protocol Buffers and Apache |
| 101 | +Avro that invented their own specialized schema language, JSON BinPack adopts a |
| 102 | +popular and industry-standard one: [JSON Schema](https://json-schema.org/). |
| 103 | +This means that you can use JSON BinPack alongside any schema evolution tooling |
| 104 | +or approach from the wider JSON Schema ecosystem. A popular one from the |
| 105 | +decentralised systems world is |
| 106 | +[Cambria](https://www.inkandswitch.com/cambria/). |
| 107 | + |
| 108 | +*** |
| 109 | + |
| 110 | +Do you have further questions or feedback? Don't hesitate in reaching out on |
| 111 | +[GitHub |
| 112 | +Discussions](https://github.com/sourcemeta/jsonbinpack/discussions)! |
| 113 | + |
| 114 | +Research |
| 115 | +-------- |
| 116 | + |
| 117 | +JSON BinPack is the result of extensive binary serialization research at the |
| 118 | +Department of Computer Science of University of Oxford. The project is led by |
| 119 | +Juan Cruz Viotti, a computer scientist with first-hand exposure to the problem |
| 120 | +of space-efficient network communication in the context of IoT and APIs, under |
| 121 | +the supervision of Mital Kinderkhedia. |
| 122 | + |
| 123 | +- [JSON BinPack: A space-efficient schema-driven and schema-less binary |
| 124 | +serialization specification based on JSON |
| 125 | +Schema](https://www.jviotti.com/dissertation.pdf) |
| 126 | + |
| 127 | + - Awarded the 2022 CAR Hoare Prize for the best performance on the project |
| 128 | + in the MSc in Software Engineering |
| 129 | + - Awarded the 2022 CAR Hoare Prize for the best performance in the |
| 130 | + examination by coursework in the MSc in Software Engineering |
| 131 | + |
| 132 | +- [Benchmarking JSON BinPack](https://arxiv.org/abs/2211.12799) |
| 133 | +- [A benchmark of JSON-compatible binary serialization |
| 134 | + specifications](https://arxiv.org/abs/2201.03051) |
| 135 | +- [A survey of JSON-compatible binary serialization |
| 136 | + specification](https://arxiv.org/abs/2201.02089) |
0 commit comments