Handling input/output binary values

[francescoingv]

I have two question about binary values: one about input and one about output.

A) About input

In the example listing following "The description of the example EchoProcess process might be:" I read:

"imagesInput": {
"title": "Inline Images Value Input",
"description": "This is an example of an image input. In this case,
the input is an array of up to 150 images that might, for example, be a set of
tiles. The oneOf[] conditional is used to indicate the acceptable image content
types; GeoTIFF and JPEG 2000 in this case. Each input image in the input array
can be included inline in the execute request as a base64-encoded string or
referenced using the link.yaml schema. The use of a base64-encoded string is
implied by the specification and does not need to be specified in the definition
of the input.",

Specifically it states: The use of a base64-encoded string is implied by the specification

Also, according to /req/core/process-execute-input-inline-binary:
The service SHALL support binary values encoded as base64-encoded strings.
However I didn't find a statement saying the input binary values must be base64-encoded strings: apparently the server must support (also) base64-encoded strings.

Question: using base64-encoded strings is a client option or is it mandatory?

B) About output

In case of a request where:

• The process description is what is provided for EchoProcess
• Negotiated execute mode: sync
• There is no return preference: Prefer HEADER = none
• There is no request for media type
• All the following results are returned by value, not by reference

B.1) Case of single output

• the output requested is only one
• the output requested is imagesOutput

"imagesOutput": {
  "schema": {
    "oneOf": [
      {
        "type": "string",
        "contentEncoding": "binary",
        "contentMediaType": "image/tiff; application=geotiff"
      },
      {
        "type": "string",
        "contentEncoding": "binary",
        "contentMediaType": "image/jp2"
      }
    ]
  }
},

B.1.1) Single item

• there is only one image to return

Question: the returned "Response Body" should either be: a) a binary value; b) a base64-encoded string ?

B.1.2) Multiple items

• there are two (or more) images to return (i.e. an array, as per Example 2 — A result encoded as JSON.)
  [Side note: I wonder if the definition of imagesOutput should include a "maxOccurs" to allow multiple items.]

Then I understand the returned "Response Body" should represent an array.
I also assume the value of each single image is returned as for the case of B.1.1)

Question: how to delimit the two images of the array in the "Response Body"?

B.2) Case of multiple output

• the output requested are two: imagesOutput (defined above in B1) ), stringOutput

I understand the "Response Body" sholud be something like:

{
  "stringOutput": "Value1",
  "imagesOutput": { "value": <image_value>, "encoding": <content_encoding>, "mediaType": <media_type> }
} 

Question: what should be <content_encoding> ? [I understand <image_value> will then be coded accordingly]

[fmigneault]

Inputs

I believe the intent is rather that, for any input whether a binary data blob or a more human-readable content that could be displayed with plain strings, {"value": "<data>"} and {"href": "<URI>"} must both be handled. Then, the specific encoding can be supplied to be explicit about any variant that you might specifically provide, eg: base32 if that is how the data source is actually encoded when embedded in the supplied JSON. However, you underlying application using that data can transform it however needed to actually perform the computation needed. Therefore, there is no "requirement" or "restriction" about the encoding to allow this flexibility.

One thing to be noted is that most media-types that employ binary data blobs suggest employing base64 when representing them in a string format (for example in TIFF: https://datatracker.ietf.org/doc/html/rfc3302#section-6). This is what the "implied" mention refers to. Even if encoding: base64 is not supplied in those cases, your implementation can assume it is the case for such media-types since it would not make sense nor be efficient to do otherwise.

---

Outputs

B.1.1)

Because it is single output with everything requested with defaults, the contents should directly be the image (i.e.: NOT within a JSON with {"value": "<data>}), and using the actual Content-Type value with binary data (as indicated by that media-type). The usual headers like Content-Transfer-Encoding can apply as well, and may be controlled in your request with Accept-Encoding for compressions on top.

The base64 applies only when embedding that content within the JSON "results" structure, which may be requested explicitly using Accept-Profile: "https://www.opengis.net/def/profile/OGC/0/ogc-results".

B.1.2) and B.2) about multiple items

maxOccurs

• That is the discussion/change of Add minOccurs and maxOccurs to output description #414
• It is now included: https://github.com/opengeospatial/ogcapi-processes/blob/master/openapi/schemas/processes-core/outputDescription.yaml#L5
• Technically, there was no "multi-item" supported before that. You could have returned many images using an embedded JSON array, but there was no consistent way to specify it (hence why maxOccurs had to be added).

Therefore, with maxOccurs omitted in that example, that example would now imply single-output (because maxOccurs: 1 by default).

[Side note: I wonder if the definition of imagesOutput should include a "maxOccurs" to allow multiple items.]

Yes. It now requires it (and should be updated in the example).

Important

There is a distinction to be made between "multiple outputs" (many {outputID} entries) and "multiple items" (the entry embeds an array or not).

If the process defines a "single output" with "multiple items", its content is negotiated directly (i.e: Accept: image/jp2 would apply to all images of that single output. If the process instead defines "multiple outputs", then outputSelection.yaml within the execution request is what dictates the media-types of each output (or default otherwise). The Accept in this case represented the "container" of all those outputs. Whether each of those outputs has single or multi-items depends on their maxOccurs.

Assuming maxOccurs>1 was indicated, the multi-item representation depends on the negotiated media-type (output selection format or Accept depending on N outputs) and/or "results" profile, depending on how many "outputID" elements the process supports.

If it is all default and >1 outputIDs are defined by the process, the Accept: application/json with "results" structure is the default with each data blob encoded as base64, since the binary cannot be directly in JSON string. Those results are either directly {"value": "..." or a nested array based on maxOccurs.

For multiple outputs, you could also request (Accept) another media-type, like multipart/related or some application/zip, which could package everything in another way relevant to that content representation.

Like the inputs, the encoding mostly depends on what the relevant media-type(s) indicate, and most of the time, the contents encoding can be inferred directly from them.

---

Personally, I prefer to indicate encoding explicitly when embedding the data blob to be clear about the intent and avoid implementations misinterpreting media-types differently between hops, and to make user that users/agents/clients not familiar with a given media-type understand it correctly, but that is an implementation-specific decision. That encoding value is consistent based on specific media-type combinations whether provided or not, and it is safe to assume that base64 is used when the data is embedded in JSON to represent what would normally be provided in a binary stream.

[francescoingv]

@fmigneault thank you for your complete answer.
Let me try to summarize it, to check I got it correctly,
and try to clarify B.1.2 which is still not clear to me.

Input

As the imagesInput schema define "type": "string", "contentEncoding": "binary"
and the input data passed with a request must be provided with a JSON format,
then the input must be coded base64.

Output B.1.1 - single output, single item

For the specific case, it will return the binary string of the image.

Output B.1.2 - single output, multiple items

According to table 11, the Content type HEADER is "as negotiated", and the Response Body is "requested process output".
In the specific case:

• the Content-type is the default (say "image/jp2"), and (as you stated) will be applied to all the images in the output array
• however it is still not clear to me what the output should be, and how to delimit the two images.

Are you suggesting in this case a "container" should be negotiated to contain the items of the single output?

Output B.2 - multiple outputs

Always according to table 11, as the Content-Type is application-json and the Response Body is returned according to results.yaml,
then imagesOutput is returned as part of a JSON, therefore the binary string (or every item of the array, made of binary strings) value is coded base64.

[fmigneault]

If the schema defines "type": "string", "contentEncoding": "binary", it means the data is represented as binary rather than plain text. Therefore, "some encoding" must happen to obtain a string representation, but which one was applied depends on the encoding parameter, and a safe default would be base64 since it is the most common and recommended one: https://datatracker.ietf.org/doc/html/rfc2045#section-6.8. Others are still allowed to submit "value": "<blob>", "encoding": "base32" or whichever other one that the string represents if that is what is needed by the specific media-type. The standard allows the flexibility just like these media-types specifications do not mandate base64 specifically, in order to support media-types that would have other restrictions. The requirement to support base64 is more to consider that you are basically guaranteed to need base64 support provided how commonly it is used, so it is a minimal interoperable format for by-value submission.

however it is still not clear to me what the output should be, and how to delimit the two images.

Assuming the example is fixed to have maxOccurs>1 or maxOccurs: unbounded, the default result for a single-output/multi-item would be in this case:

{
  "imagesOutput": [
    {"value": "<blob1>", "encoding": "base64", "mediaType": "image/jp2"},
    {"value": "<blob2>", "encoding": "base64", "mediaType": "image/jp2"}
  ]
}

since image/jp2 (https://datatracker.ietf.org/doc/html/rfc3745#section-4.1) indicates "base64 is a suitable encoding"

The "as negotiated" is to reflect that Accept: multipart/related allows you to return instead (non-default):

Content-Type: multipart/related; boundary=next_part

--next_part
Content-Type: image/jp2
Content-Transfer-Encoding: base64

<blob1>
--next_part
Content-Type: image/jp2
Content-Transfer-Encoding: base64

<blob2>
--next_part--

Or even Accept: application/zip that wraps the 2 images (in such case, no base64 encoding would happen, since the image files are themselves the binary blobs).

The only required "container" is the JSON results.yaml, but others like the ones above are allowed at the discretion of the implementation.

[pvretano]

@fmigneault you type way faster that I do! LOL! But, yes, what @fmigneault said...

[francescoingv]

@fmigneault thank you.
Then I would suggest to add the above clarification within table 11 for the case of single output request, where the requested output has maxOccours > 1

[pvretano]

@francescoingv yes, I will add it the list of changes I am making for the RFC submission ...