proposal: thing-model-catalog: create a catalog of Thing Models

[hadjian]

The Thing Model Catalog

The goal of this proposal is to motivate the development of a Thing Model Catalog. The catalog shall enable the collection of Thing Models for fixed function devices, provided and curated by the community. This will enable users of the catalog to share the low-value efforts in interfacing with devices that lack self-description in a standardized way.

Consumption

We target the use case of searching for Thing Models for devices to be onboarded to arbitrary WoT consumers. We identified three simple workflows:

1. Manual Browsing: A system integrator searches the catalog manually and finds a matching Thing Models. The Thing Model is downloaded manually, Thing Descriptions are generated from the TM and supplied to a WoT consumer. The WoT consumer is now able to establish communication to the devices.

2. WoT Consumer Integration: A WoT consumer that offers integration with the catalog interface offers browsing, searching and filtering with a custom user interface. Via this user interface, the user selects a Thing Model from the query results, specifies replacement parameters to convert the Thing Model to Thing Descriptions and onboards devices to the WoT consumer.

3. Bundling with a Consumer: A connectivity solution provides the Thing Model Catalog as an integrated part of its offering, i.e. will publish a curated copy of the catalog with the solution.

4. Programmatic Consumption: A connectivity solution offers autodiscovery of devices and searches the catalog for matching Thing Models to be loaded for instantiation.

Contribution

To enable the contribution of Thing Models, the catalog needs to define a contribution interface. This interface needs to enable the following traits:

1. Provenance: A catalog shall make the ownership and authority of contributions apparent. A catalog shall provide hosting infrastructure, but the hoster shall not be responsible for all contributions. These need to be owned and managed by contributors.

2. Quality Gate: The interface shall act as a quality gate to enforce a minimum standard for contributed Thing Models, as there can be different levels of quality. The catalog shall communicate this quality level to consumers.

3. Metadata: The catalog shall support other data in addition to the Thing Models, such as author information, rating, handbook, product image etc.

4. Context Extensions: The catalog shall allow custom extensions to Thing Models, which are not checked for validity in a first release.

5. Fast Feedback Loop: A contributor shall be able to execute the same tests locally, which are required for a contribution to pass to be accepted by a remote catalog.

How

This section lists requirements that need to be addressed in an implementation without proposing a technical solution.

Storage Schema

The storage schema, e.g. directory structure and file name convention in case of a file system needs to allow for human browsability as well as programmatic processing.

The storage schema might allow for a contributor designed hierarchical layout for his/her contributions.

The storage schema must allow grouping of related files, e.g. handbook, image, metadata file.

Identification

A user must be able to uniquely identify a Thing Model. As the Thing Description standard does not enforce a specific ID schema, the catalog must generate unique IDs for Thing Models.

Quality Stages

We currently see the following quality stages:

Stage Name	Description
Syntactic Validation	TM validates against official JSON Schema
Semantic Validation	JSON-LD shape is validated
Runtime Validation	TM was used to interact with a device and device responses validate against data schema in TM
Human Validation	Values returned from device correspond to expected values, e.g. strings are decoded correctly
Connectivity Solution Compatibility	List of runtimes, with which the TM was tested
Rating	Means of expressing perceived quality by users

Allow for Duplicates

A catalog must accept multiple Thing Models for the same device, as multiple users might author them independently. A consumer can decide which one to trust and can get a hint by the quality stages passed by the different Thing Models.

Access Restrictions

Contributions shall be owned by one user, who will have the exclusive right to change a Thing Model. The catalog shall enable proposed changes by other users to be accepted by the owning user.

Offline / Online Usage

It must be possible to use a snapshot of an online catalog in an offline scenario

Cross Catalog Search

It must be possible to search across multiple catalogs to enable the concurrent use of a private and a public catalogs.

Needs Discussion

• Support Thing Model Compilation for composed or inheriting Thing Models
• Instance creation from Thing Models to Thing Descriptions

cc => @hadjian @andrisciu

[ondrejtomcik]

Hi @hadjian. Few proposals on the content:

A system integrator searches the catalog manually and finds a matching TM. The TM is downloaded manually, TDs are generated from the TM and used to load the parameters into the connectivity solution and the communication to the device can be established.

There's potential for misunderstanding here. I suggest rephrasing it to: A system integrator accesses the WoT Consumer's user interface and searches for the appropriate TM using various filters, such as manufacturer name, product name, or other relevant identifiers. Upon finding a match - WoT Consumer getting the link to matching TDs from the catalogue, the WoT Consumer fetches the TM. After that the WoT Consumer prompts the user for the necessary information to convert the TM to the TD.

A connectivity solution provides the Thing Model Catalog as an integrated part of its offering, i.e. will publish a curated copy of the catalog with the solution.

The WoT Consumer's can integrate the snapshot of the catalogue for offline search in case the solution is disconnected from the internet.

"A connectivity solution"

I would propose to use the WoT terminology here - WoT Consumer.

It must be possible to search across multiple repositories to enable the concurrent use of a private and a public repository.

Repository is a new term in this document. I think you wanted to use catalogue.

[hadjian]

Thx @ondrejtomcik for your input.

A system integrator searches the catalog manually and finds a matching TM. The TM is downloaded manually, TDs are generated from the TM and used to load the parameters into the connectivity solution and the communication to the device can be established.

There's potential for misunderstanding here. I suggest rephrasing it to: A system integrator accesses the WoT Consumer's user interface and searches for the appropriate TM using various filters, such as manufacturer name, product name, or other relevant identifiers. Upon finding a match - WoT Consumer getting the link to matching TDs from the catalogue, the WoT Consumer fetches the TM. After that the WoT Consumer prompts the user for the necessary information to convert the TM to the TD.

You are right that it doesn't seem to be clear, as I was thinking of a different scenario. And the one you are describing is actually missing. Scenario 1. is supposed to describe a scenario where there is no integration between the catalog and a WoT consumer, so a user needs to be able to browse the catalog manually, download a file, replace the parameters and load it into the WoT consumer by any means the WoT consumer offers to load TDs.

I will add the integrated scenario and try to rephrase the first scenario.

A connectivity solution provides the Thing Model Catalog as an integrated part of its offering, i.e. will publish a curated copy of the catalog with the solution.

The WoT Consumer's can integrate the snapshot of the catalogue for offline search in case the solution is disconnected from the internet.

I don't quite understand the difference to what I wrote. I meant that the WoT consumer will integrate a snapshot of the catalog when it is released to be used in an offline scenario. So the dev team of the WoT consumer download the snapshot, curate it and bundle it with their consumer. The consumer can then use the catalog without internet access.

"A connectivity solution"

I would propose to use the WoT terminology here - WoT Consumer.

Agree.

It must be possible to search across multiple repositories to enable the concurrent use of a private and a public repository.

Repository is a new term in this document. I think you wanted to use catalogue.

Agree.

[hadjian]

• Added labels for the scenarios to be more clear
• Added scenario WoT Consumer Integration
• Removed abbreviations
• Replaced connectivity solution with WoT consumer

[hadjian]

• Added a quality stage for Semantic Validation

@mkovatsc You are deep into that topic. Our plan is to allow context extensions and custom terms, but will not validate in a first release. Do you see any obstacle to integrate semantic validation in a future release?

[hadjian]

Added titles to the requirements in Contribution section.

[hadjian]

Notes for proposal review:

• Roles: Device Manufacturers, Connectivity Solutions Providers, Integrators, Domain Experts
• Mandatory metadata for browsing and searching (author, manufacturer, mpn)
• Directory structure
• ID (close to dtmi)
• TOC for simple catalog (use forges or simple HTTP server)
• Namespaces for quick contributions
• Official manufacturer namespaces