Skip to content

V0.0.2: add effective inserts and deletions #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,14 +69,41 @@ The exported property contains an array of definitions, each linking a match to
- `options.resourceFormat`: Version format describing the format of the contents. Keys may be added to this format, but they may not be removed. Filter the properties as needed.
- `options.gracePeriod`: Only send the response after a certain amount of time. This will group changes in the future.
- `options.ignoreFromSelf`: Don't inform about changes that originated from the microservice to be informed (based on the hostname).
- `options.matchOnEffective`: Only match with effective inserts and deletions, not all inserts and deletions. Default: false.
- `options.requestPerMuCallIdTrail`: Execute one request per muCallIdTrail. mu-authorization batches multiple requests, which might have different idTrails.

## Delta formats

The delta may be offered in multiple formats. Versions should match the exact string. Specify `options.resourceFormat` to indicate the specific resourceformat.

#### v0.0.2

v0.0.2 is the last format added. It makes a distinction between all inserts and deletions and effective inserts and deletions. An inserts is effective if the triple is not yet in the triplestore. A deletion is effective if the triple was present in the triplestore.

This version uses an updated mu-authorization that batches inserts and deletions together to improve performance. This imposes the need for an `index` field. This `index` is incremental starting from Unix time in milliseconds.

```json
[
{ "inserts": [{"subject": { "type": "uri", "value": "http://mu.semte.ch/" },
"predicate": { "type": "uri", "value": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" },
"object": { "type": "uri", "value": "https://schema.org/Project" }},
{"subject": { "type": "uri", "value": "http://mu.semte.ch/" },
"predicate": { "type": "uri", "value": "http://purl.org/dc/terms/modified" },
"object": { "type": "literal", "value": "https://schema.org/Project", "datatype": "http://www.w3.org/2001/XMLSchema#dateTime"}}],
"deletes": [],
"effectiveInserts": [{"subject": { "type": "uri", "value": "http://mu.semte.ch/" },
"predicate": { "type": "uri", "value": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type" },
"object": { "type": "uri", "value": "https://schema.org/Project" }}],
"effectiveDeletions": [],
"index" :1629367246152 }
]
```



### v0.0.1

v0.0.1 is the latest format of the delta messages. It may be extended with authorization rights etc. in the future. The value encoding follows the [json-sparql spec RDF term encoding](https://www.w3.org/TR/sparql11-results-json/#select-encode-terms). For example:
v0.0.1 may be extended with authorization rights etc. in the future. The value encoding follows the [json-sparql spec RDF term encoding](https://www.w3.org/TR/sparql11-results-json/#select-encode-terms). For example:

```json
[
Expand All @@ -95,7 +122,7 @@ v0.0.1 is the latest format of the delta messages. It may be extended with autho
Genesis format as described by the initial Delta service PoC. It looks like:

```json
{
{
"delta": {
"inserts": [{"s": "http://mu.semte.ch/",
"p": "http://www.w3.org/1999/02/22-rdf-syntax-ns#type",
Expand All @@ -115,7 +142,7 @@ Debugging can be enabled in the service by setting environment variables. The f
- `DEBUG_DELTA_SEND`: Logs all delta messages that are being sent to clients
- `DEBUG_DELTA_MATCH`: Logs a check for each target block, indicating a check will occur
- `DEBUG_TRIPLE_MATCHES_SPEC`: Extensive logging for triples matching a given specification. Handy when requests are unexpectedly not sent.

## Extending

You are encouraged to help figure out how to best extend this service. Fork this repository. Run an experiment. Open an issue or PR describing your experiment. Feel free to open up an issue if you would like to discuss a possible extension.
Loading