v3.??? Data vs Serialized Examples in Example Objects #4647

handrews · 2025-06-02T19:34:40Z

This is most of what we would do for the first and most essential of three Example Object related proposals I went over in the last TDC call. I'm posting it because:

I cannot remember the last time I learned this much writing a PR. Using these new example fields helped me find errors in the examples and think through the spec and user implications more clearly. I think this shows the tremendous value these changes would bring compared to the confusing an inconsistently-supported status quo.
This change fits the 3.2 theme of better data modeling and serialization. Without these changes, the other changes we are adding are harder to understand and use correctly. With these changes, most complex modeling and serializaton scenarios can now be shown clearly (the others require the other two parts of this overall proposal, but we could take just this part and not those and still add tremendous value).

Please look through this and particularly focus on how this allowed me to change our in-spec examples to make many things more clear, and show how OAD authors can similarly make their intent clear.

Note that the changes to the XML examples don't (I think) assume the nodeType change in PR #4592, but I might have lost track of that if you see something strange there. The point is that this helps tremendously in showing expected XML behavior.

schema changes are included in this pull request
schema changes are needed for this pull request but not done yet
no schema changes are needed for this pull request

src/oas.md

handrews · 2025-06-04T19:08:52Z

@mikekistler fixed that example- it had been done with integers but those aren't interesting serialization cases! So I switched to strings.

I also added some text about it being easier/less ambiguous to parse the serialized example and validate the data in many cases than to try to go the other way around.

src/oas.md

handrews · 2025-06-04T22:53:18Z

@mikekistler I have clarified the serialization rules in the latest added commit.

mikekistler

This will help clarify what examples are and how they should be used.

My comments are intended to clarify some points and also to make this change a more evolutionary transition.

mikekistler · 2025-06-05T00:54:42Z

src/oas.md

+    examples:
+      oneMinute:
+        dataValue: 60
+        serializedValue: 'X-Rate-Limit-Reset: 60'


I love that you have added all these examples!

Honestly it was one of the best things about writing the PR and is what convinced me it's worth making a strong case for it.

mikekistler · 2025-06-05T00:57:38Z

src/oas.md


 ##### Fixed Fields

 | Field Name | Type | Description |
 | ---- | :----: | ---- |
 | <a name="example-summary"></a>summary | `string` | Short description for the example. |
 | <a name="example-description"></a>description | `string` | Long description for the example. [CommonMark syntax](https://spec.commonmark.org/) MAY be used for rich text representation. |
-| <a name="example-value"></a>value | Any | Embedded literal example. The `value` field and `externalValue` field are mutually exclusive. To represent examples of media types that cannot naturally represented in JSON or YAML, use a string value to contain the example, escaping where necessary. |
-| <a name="example-external-value"></a>externalValue | `string` | A URI that identifies the literal example. This provides the capability to reference examples that cannot easily be included in JSON or YAML documents. The `value` field and `externalValue` field are mutually exclusive. See the rules for resolving [Relative References](#relative-references-in-api-description-uris). |
+| <a name="example-data-value"></a>dataValue | Any | An example of the data structure that MUST be valid according to the relevant [Schema Object](#schema-object).  If this field is present, `externalDataValue`, `value`, and `externalValue` MUST be absent. |


Since JSON Schemas validate "instances", should we use "instance" instead of "data structure"?

I couldn't decide which language was better. Easy to tweak at any point- I'll think about it as I do the other updates. [EDIT: I decided "instance" was too JSON Schema jargon-y, plus we go outside of the JSON Schema instance model with things like raw binary data.]

src/oas.md

mikekistler · 2025-06-05T01:23:46Z

src/oas.md


-##### Example Object Examples
+Historically, the Example Object's `value` and `externalValue` field and the non-Schema Object singular `example` fields were intended to show examples of the serialized form, while allowing JSON or YAML examples to be included inline rather than as serialized strings.


I can't speak to the "original intent" for these fields -- I wasn't involved at the time. But I wonder if this was really consciously decided as the intent, and then somehow omitted from the spec, or if instead it was not really understood that there is a difference between the data value and serialized value and that is why the distinction was not included in the spec.

@mikekistler I understand that tooling vendors have implemented it differently, but these quotes from 3.0.3 seem pretty clear to me, and we've debated this before and always agreed that the encoded/serialized form was the intent. AFAIK, this effort by you right now is the first time anyone deeply involved has taken the opposite position in terms of what the wording means:

From the Parameter Object:

When example or examples are provided in conjunction with the schema object, the example MUST follow the prescribed serialization strategy for the parameter.

From the Media Type Object example field:

The example object SHOULD be in the correct format as specified by the media type.

From the Media Type Object examples field:

Examples of the media type. Each example object SHOULD match the media type and specified schema if present.

From the Example Object value field:

To represent examples of media types that cannot naturally represented in JSON or YAML, use a string value to contain the example, escaping where necessary.

From the Example Object externalValue field:

This provides the capability to reference examples that cannot easily be included in JSON or YAML documents.

From the Example Object after the fixed fields table:

In all cases, the example value is expected to be compatible with the type schema of its associated value. Tooling implementations MAY choose to validate compatibility automatically, and reject the example value(s) if incompatible.

I honestly don't see how all of this above, taken together, can be interpreted in any other way.

mikekistler · 2025-06-05T01:30:48Z

src/oas.md

+          {
+            "author": "A. Writer",
+            "title": "An Older Book",
+            "rating": 4.5
+          }


Should this serialized value be a string, which condensed white-space (as was done above)?

@mikekistler it is a string, that's what the | at the end of the serializedValue: | does (YAML block literal).

As for minimizing, I did that for the parameter because it has to be shoved into the query string. This is presumed to be for a body, so it might be sent without that minimizing. But TBH I was just trying out all sorts of different things, I'm not super-attached to this.

src/oas.md

handrews

I'll work on an update hopefully tonight, or at least before the meeting tomorrow morning.

src/oas.md

handrews · 2025-06-05T02:51:32Z

src/oas.md

+          {
+            "author": "A. Writer",
+            "title": "An Older Book",
+            "rating": 4.5
+          }


@mikekistler it is a string, that's what the | at the end of the serializedValue: | does (YAML block literal).

As for minimizing, I did that for the parameter because it has to be shoved into the query string. This is presumed to be for a body, so it might be sent without that minimizing. But TBH I was just trying out all sorts of different things, I'm not super-attached to this.

src/oas.md

handrews · 2025-06-05T03:00:22Z

src/oas.md


 ##### Fixed Fields

 | Field Name | Type | Description |
 | ---- | :----: | ---- |
 | <a name="example-summary"></a>summary | `string` | Short description for the example. |
 | <a name="example-description"></a>description | `string` | Long description for the example. [CommonMark syntax](https://spec.commonmark.org/) MAY be used for rich text representation. |
-| <a name="example-value"></a>value | Any | Embedded literal example. The `value` field and `externalValue` field are mutually exclusive. To represent examples of media types that cannot naturally represented in JSON or YAML, use a string value to contain the example, escaping where necessary. |
-| <a name="example-external-value"></a>externalValue | `string` | A URI that identifies the literal example. This provides the capability to reference examples that cannot easily be included in JSON or YAML documents. The `value` field and `externalValue` field are mutually exclusive. See the rules for resolving [Relative References](#relative-references-in-api-description-uris). |
+| <a name="example-data-value"></a>dataValue | Any | An example of the data structure that MUST be valid according to the relevant [Schema Object](#schema-object).  If this field is present, `externalDataValue`, `value`, and `externalValue` MUST be absent. |


I couldn't decide which language was better. Easy to tweak at any point- I'll think about it as I do the other updates. [EDIT: I decided "instance" was too JSON Schema jargon-y, plus we go outside of the JSON Schema instance model with things like raw binary data.]

handrews · 2025-06-05T03:01:03Z

src/oas.md

+    examples:
+      oneMinute:
+        dataValue: 60
+        serializedValue: 'X-Rate-Limit-Reset: 60'


Honestly it was one of the best things about writing the PR and is what convinced me it's worth making a strong case for it.

src/oas.md

handrews · 2025-06-05T03:45:16Z

src/oas.md


-##### Example Object Examples
+Historically, the Example Object's `value` and `externalValue` field and the non-Schema Object singular `example` fields were intended to show examples of the serialized form, while allowing JSON or YAML examples to be included inline rather than as serialized strings.


@mikekistler I understand that tooling vendors have implemented it differently, but these quotes from 3.0.3 seem pretty clear to me, and we've debated this before and always agreed that the encoded/serialized form was the intent. AFAIK, this effort by you right now is the first time anyone deeply involved has taken the opposite position in terms of what the wording means:

From the Parameter Object:

When example or examples are provided in conjunction with the schema object, the example MUST follow the prescribed serialization strategy for the parameter.

From the Media Type Object example field:

The example object SHOULD be in the correct format as specified by the media type.

From the Media Type Object examples field:

Examples of the media type. Each example object SHOULD match the media type and specified schema if present.

From the Example Object value field:

To represent examples of media types that cannot naturally represented in JSON or YAML, use a string value to contain the example, escaping where necessary.

From the Example Object externalValue field:

This provides the capability to reference examples that cannot easily be included in JSON or YAML documents.

From the Example Object after the fixed fields table:

In all cases, the example value is expected to be compatible with the type schema of its associated value. Tooling implementations MAY choose to validate compatibility automatically, and reject the example value(s) if incompatible.

I honestly don't see how all of this above, taken together, can be interpreted in any other way.

handrews · 2025-06-05T04:32:11Z

@mikekistler I have updated quite a few things, and left other things the same, with the upshot being that this PR is only about adding new fields to the Example Object and showing their use:

I have un-deprecated all of the fields, and done my best to retain the prior language around them- some language moved from just after the bit about x- extensions to right at the top of the "Working With Examples" section because otherwise it was just too awkward
This means keeping the "override" language- we can talk about changing that separately as it is completely independent of whether we add new fields
This also means not providing new guidance on how to interpret the old fields, as that needs to be done whether we add new fields or not
I decided against using "instance" as that is too jargon-y, and more a JSON Schema thing, and arguably we work with data beyond the JSON Schema instance model anyway (e.g. raw binary). If we decide to add the fields we can debate the language, but let's decide whether the fields are worth adding first, please.
I did fix the XML examples and various other small things.
I'm probably forgetting something, I'll check again in the morning

lornajane · 2025-06-05T16:47:41Z

There's a lot of support for this change and especially for the examples that are added. We need to discuss naming before merging, but we think this can go into 3.2.

src/oas.md

Co-authored-by: Dan Hudlow <[email protected]>

handrews · 2025-06-05T19:55:45Z

@OAI/tsc and all, I have filed several issues to split out various related conversations and keep this PR moving to conclusion:

v3.2: Decide on names of new Example Object fields #4658
- @mikekistler @lornajane I have added this as a release-blocking issue to Prepare 3.2.0 and 3.1.2 releases #4595 so that we can merge this PR with the current names while we keep debating the permanent names. Let's not keep this held open once we agree on the behvior
v3.1+: Should new guidance be given for example and the Example Object's value and externalValue? #4659
v3.2: Should the value and externalValue fields be deprecated? #4660
v3.2: Should the example field be deprecated (Parameter, Header, and Media Type Objects) #4661
v3.1: Should we change the "?" in the style example table (again)? #4662
v3.1+: Should we change how non-schema examples "override" schema examples? #4664

karenetheridge · 2025-06-08T18:41:47Z

src/oas.md

+examples:
+  number:
+    dataValue: 12345678
+    serializedValue: "token: 12345678"


Can we add a second example, demonstrating the use of an array coming from deserializing the simple style?

Suggested change

examples:

number:

dataValue: 12345678

serializedValue: "token: 12345678"

examples:

one number:

dataValue: 12345678

serializedValue: "token: 12345678"

two numbers:

dataValue:

- 12345678

- 99999999

serializedValue: "token: 12345678,99999999"

@karenetheridge as noted in the past TDC call, I'd like to handle newly added examples separately, otherwise we'll never finish.

@karenetheridge upon further thought, I'm going to split this PR up to separate the new fields from all of the extra example updates. I'll take this request into consideration then- I think that might be better than either allowing endless new example requests here or trying to keep them out of this process entirely.

karenetheridge · 2025-06-08T18:45:44Z

src/oas.md

      # JSON Text Sequences require an unprintable character
      # that cannot be escaped in a YAML string, and therefore
      # must be placed in an external document shown below
-      externalValue: examples/log.json-seq
+      externalSerializedValue: examples/log.json-seq


It would be really cool if we had the html renderer detect this keyword and add a hyperlink to a real file named "./examples/log.json-seq" in the github-pages repo.

src/oas.md

Co-authored-by: Karen Etheridge <[email protected]>

hudlow · 2025-06-09T15:09:12Z

src/oas.md


 ##### Fixed Fields

 | Field Name | Type | Description |
 | ---- | :----: | ---- |
 | <a name="example-summary"></a>summary | `string` | Short description for the example. |
 | <a name="example-description"></a>description | `string` | Long description for the example. [CommonMark syntax](https://spec.commonmark.org/) MAY be used for rich text representation. |
+| <a name="example-data-value"></a>dataValue | Any | An example of the data structure that MUST be valid according to the relevant [Schema Object](#schema-object).  If this field is present, `externalDataValue`, `value`, and `externalValue` MUST be absent. |
+| <a name="example-external-data-value"></a>externalDataValue | `string` | A URI that identifies the data example in a separate document, allowing for values not easily expressed in JSON or YAML.  This is usually only needed when working with binary data. The value MUST be valid according to the relevant Schema Object. If this field is present, then `dataValue`, `value`, and `externalValue` MUST be absent. See also the rules for resolving [Relative URI References](#relative-references-in-api-description-uris). |


Aside from some clarifications that could be postponed to another PR, my big concern is that I can't make heads or tails out of what a non-serialized external value looks like. In my mind, external values are inherently serialized: if you can provide externalDataValue: ./image.png and the image is treated opaquely then it implies that when you provide externalDataValue: ./data.json then the JSON document will be treated opaquely, and this would semantically match dataValue: "{ ... }" not dataValue: { ... }.

If you wanted the latter then the phrase "allowing for values not easily expressed in JSON or YAML" doesn't seem right, plus if all you want is to move a JSON data structure to an external URI, you can just use $ref at the usage point for the example value? Yes, that means the whole Example Object is externalized but... that seems fine?

It is my belief that we should merely clarify that externalValue is treated opaquely (i.e., as a serialized value) and skip adding externalDataValue and externalSerializedValue.

externalDataValue acts just like the $ref keyword -- the serialization level of the file is the same as the referencing document, but it's in a separate file solely because of its size (presumably), or to perhaps allow it to be modified at a different cadence than the referencing document itself (although that makes less sense here when the example needs to align with the schema).

I would imagine it would be the least-used of the four new keywords, but I see no reason to omit it.

@karenetheridge so you disagree with the text as written in this table and with the example on line 2328? If it works like this, why not just allow dataValue to be a reference?

Because then you can't have an example that uses a property named $ref, this has been discussed extensively in past issues, search for an issue allowing for combining examples.

src/oas.md

hudlow · 2025-06-09T15:57:29Z

src/oas.md

+          }
+```
+
+###### Binary Examples


I think it might also be nice to show a data-URL-scheme example. My assumption is that what URL schemes an interpreter of an OpenAPI description supports are completely out of scope, but there's no reason one couldn't support the data scheme?

Something like:

externalSerializedValue: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAIAAAACCAIAAAD91JpzAAAABGdBTUEAALGPC_xhBQAAADhlWElmTU0AKgAAAAgAAYdpAAQAAAABAAAAGgAAAAAAAqACAAQAAAABAAAAAqADAAQAAAABAAAAAgAAAADO0J6QAAAAEElEQVQIHWP8zwACTGCSAQANHQEDqtPptQAAAABJRU5ErkJggg%3D%3D"`

@hudlow can you please file an issue requesting that the OAS support this for external example fields (regardless of which external fields we end up having)? I think it is an interesting idea worth discussing but orthogonal to this PR.

Done: #4674

Co-authored-by: Dan Hudlow <[email protected]>

handrews · 2025-06-09T18:32:19Z

I am closing this in favor of the following PRs, because this is too big and the conversation has a lot of complex threads that have already been resolved.

v3.2: Add data vs serialized Example Object fields (Revised) #4671 (adds fields- @hudlow I made a section about encodings)
v3.2: Example Object example updates for new fields #4672 (Example Object example updates- these messed up the diff if included in 4671)
v3.2: Explain Param/Header/Enc example serialization #4673 (Parameter/Header style serialization rules- I was unsure of these on the last call so I thought through the rules more and propose these)
Numerous forthcoming PRs updating examples for Parameter, Header, Request Body, Response, Media Type, Encoding, and XML Objects

handrews added param serialization Issues related to parameter and/or header serialization media and encoding Issues regarding media type support and how to encode data (outside of query/path params) example obj/keywords Issues with the Example Object or exampel(s) keywords labels Jun 2, 2025

handrews added 2 commits June 2, 2025 12:47

First pass at more example fields

1f19142

Schema changes for Example Obj changes

caa302a

handrews force-pushed the examples branch from 878b338 to caa302a Compare June 2, 2025 19:47

This was referenced Jun 2, 2025

v3.1.2: Style examples cannot include ? #4627

Closed

v3.2: Allow Parameter/Header "examples" field with "content" field #4648

Open

mikekistler reviewed Jun 4, 2025

View reviewed changes

src/oas.md Show resolved Hide resolved

Fix an example, add more guidance on validation

f76be56

handrews mentioned this pull request Jun 4, 2025

Open Community (TDC) Meeting, Thursday 05 June 2025 #4642

Open

mikekistler reviewed Jun 4, 2025

View reviewed changes

src/oas.md Show resolved Hide resolved

Clarify serialization scope, fix typos

51c1270

mikekistler requested changes Jun 5, 2025

View reviewed changes

handrews commented Jun 5, 2025

View reviewed changes

Un-deprecate fields

6244324

hudlow reviewed Jun 5, 2025

View reviewed changes

src/oas.md Outdated Show resolved Hide resolved

Fix example field usage

e22bf84

Co-authored-by: Dan Hudlow <[email protected]>

handrews mentioned this pull request Jun 6, 2025

v3.1+: Should we change how non-schema examples "override" schema examples? #4664

Open

karenetheridge reviewed Jun 8, 2025

View reviewed changes

src/oas.md Outdated Show resolved Hide resolved

Fix typo

49680e6

Co-authored-by: Karen Etheridge <[email protected]>

hudlow suggested changes Jun 9, 2025

View reviewed changes

hudlow reviewed Jun 9, 2025

View reviewed changes

src/oas.md Outdated Show resolved Hide resolved

hudlow reviewed Jun 9, 2025

View reviewed changes

Fix example

aceccab

Co-authored-by: Dan Hudlow <[email protected]>

This was referenced Jun 9, 2025

v3.2: Add data vs serialized Example Object fields (Revised) #4671

Open

v3.2: Explain Param/Header/Enc example serialization #4673

Open

handrews closed this Jun 9, 2025


		##### Example Object Examples
		Historically, the Example Object's `value` and `externalValue` field and the non-Schema Object singular `example` fields were intended to show examples of the serialized form, while allowing JSON or YAML examples to be included inline rather than as serialized strings.

v3.??? Data vs Serialized Examples in Example Objects #4647

v3.??? Data vs Serialized Examples in Example Objects #4647

Uh oh!

Conversation

handrews commented Jun 2, 2025

Uh oh!

Uh oh!

handrews commented Jun 4, 2025

Uh oh!

Uh oh!

handrews commented Jun 4, 2025

Uh oh!

mikekistler left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

handrews Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

handrews left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

handrews Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

handrews commented Jun 5, 2025

Uh oh!

lornajane commented Jun 5, 2025

Uh oh!

Uh oh!

handrews commented Jun 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

karenetheridge Jun 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

handrews Jun 5, 2025 •

edited

Loading

handrews Jun 5, 2025 •

edited

Loading

handrews commented Jun 5, 2025 •

edited

Loading

karenetheridge Jun 8, 2025 •

edited

Loading

hudlow Jun 9, 2025 •

edited

Loading