feat(concertino): optimize format and built target (#1048)

* feat(concertino): optimize format and built target

Signed-off-by: mttrbrts <[email protected]>

* docs(README): add replit link

Signed-off-by: mttrbrts <[email protected]>

---------

Signed-off-by: mttrbrts <[email protected]>

Author: mttrbrts

packages/concertino/README.md

@@ -2,6 +2,15 @@
 
 Concertino is a lightweight variant of the Concerto metamodel format, optimized for client applications that need to programmatically introspect declarations without taking a dependency on the full Concerto SDK.
 
+## Benefits of the Concertino Format
+The Concertino format provides several advantages:
+
+- Flatter structure: By denormalizing the inheritance hierarchy, consumers don't need to traverse the hierarchy to understand a definition.
+
+- More explicit metadata: Additional metadata makes it clearer how properties should be interpreted.
+
+- Ready for consumption: The transformation prepares the data for easier consumption by tools and renderers, without requiring additional processing.
+
 ## Features
 
 - **Fully Resolved Type References**: No ambiguous shortnames
@@ -10,14 +19,10 @@ Concertino is a lightweight variant of the Concerto metamodel format, optimized
 - **Extended Inheritance Chain**: Concepts list their full inheritance chain, not just immediate parent
 - **Scalar Type Denormalization**: For convenience in client applications
 - **Strict Mode By Default**: Namespaces are always versioned.
+- **Support for Partial Models** Allowing client applications to filter models to tailor payloads for their use cases.
+- **Lossless Conversion** Concertino is designed for 100% lossless roundtrip conversion with Concerto models.
 
-## Lossless Conversion
-
-Concertino is designed for 100% lossless roundtrip conversion:
-
-Concerto (Metamodel) → Concertino → Concerto (Metamodel)
-
-Note that namespaces in the source Concerto model should be fully resolved (including for local type references).
+> Concerto (Metamodel) → Concertino → Concerto (Metamodel)
 
 ## Installation
 
@@ -29,37 +34,39 @@ npm install @accordproject/concertino
 
 ### Using the ConcertinoConverter Class
 
+[Try it in Replit](https://replit.com/@mttrbrts/AccordProjectConcertino?v=1)
 ```javascript
+const { ModelManager } = require('@accordproject/concerto-core');
 const { ConcertinoConverter } = require('@accordproject/concertino');
 
-// Create a converter with custom options
-const converter = new ConcertinoConverter({
-  version: '0.1.0-alpha.3' // Specify concertino version
-});
+// Prepare the Concerto model
+const mm = new ModelManager();
+mm.addModel(MODEL_FILE_CONTENTS);
+const model = mm.getModelFile(MODEL_FILE_NAMESPACE).getAst();
+
+// Initialize Concertino
+const converter = new ConcertinoConverter();
+const models = { models: [model] };
 
 // Convert from Concerto metamodel to Concertino
-const concertino = converter.fromConcertoMetamodel(concertoMetamodel);
+const concertino = converter.fromConcertoMetamodel(models);
+console.log('#### Concertino:')
+console.log(JSON.stringify(concertino, null, 2));
+console.log();
 
 // Convert from Concertino back to Concerto metamodel
 const metamodel = converter.toConcertoMetamodel(concertino);
-
+console.log('#### Concerto AST (Full Metamodel Instance):')
+console.log(JSON.stringify(metamodel, null, 2));
 ```
 
-### Using Individual Functions
-
-```javascript
-const { convertToConcertino, convertToMetamodel } = require('@accordproject/concertino');
-
-// Convert from Concerto metamodel to Concertino
-const concertino = convertToConcertino(concertoMetamodel);
+## Model Size
 
-// Convert from Concertino back to Concerto metamodel
-const metamodel = convertToMetamodel(concertino);
-```
+Despite the denormalization of metadata, the JSON serialization of Concertino models are often smaller in size than their Concerto AST equivalents due to a flatter, dictionary-like design and the removal of type-discriminators (i.e. `$class` properties).
 
-## File Size Reduction
+It is expected that models that make heavy use of inheritance would be larger than their equivalent Concerto AST.
 
-Basic testing with pretty-printed, uncompressed files shows a 70-80% reduction in file size without loss of expressiveness.
+Note that when converting models, the namespaces in the source Concerto model should be fully resolved (including for local type references).
 
 ## Example Concertino JSON Format
 
@@ -69,36 +76,27 @@ Below is an example of how a simple Concerto model is represented in the Concert
 {
   "declarations": {
     "[email protected]": {
-      "name": {
-        "localName": "Address",
-        "namespace": "readme",
-        "version": "1.0.0",
-      },
       "properties": {
         "city": {
           "name": "city",
-          "scalarType": "String",
           "type": "String",
           "vocabulary": {
             "label": "City/Town",
           },
         },
         "country": {
           "name": "country",
-          "scalarType": "String",
           "type": "String",
         },
         "street": {
           "name": "street",
-          "scalarType": "String",
           "type": "String",
           "vocabulary": {
             "label": "Street Address",
           },
         },
         "zipCode": {
           "name": "zipCode",
-          "scalarType": "String",
           "type": "String",
         },
       },
@@ -108,20 +106,10 @@ Below is an example of how a simple Concerto model is represented in the Concert
       },
     },
     "[email protected]": {
-      "name": {
-        "localName": "Email",
-        "namespace": "readme",
-        "version": "1.0.0",
-      },
       "regex": "/^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}$/",
       "type": "StringScalar",
     },
     "[email protected]": {
-      "name": {
-        "localName": "Person",
-        "namespace": "readme",
-        "version": "1.0.0",
-      },
       "properties": {
         "address": {
           "isOptional": true,
@@ -138,7 +126,6 @@ Below is an example of how a simple Concerto model is represented in the Concert
             0,
             null,
           ],
-          "scalarType": "Integer",
           "type": "Integer",
         },
         "email": {
@@ -153,15 +140,13 @@ Below is an example of how a simple Concerto model is represented in the Concert
         },
         "firstName": {
           "name": "firstName",
-          "scalarType": "String",
           "type": "String",
           "vocabulary": {
             "label": "Given Name",
           },
         },
         "lastName": {
           "name": "lastName",
-          "scalarType": "String",
           "type": "String",
           "vocabulary": {
             "label": "Family Name",
@@ -178,7 +163,7 @@ Below is an example of how a simple Concerto model is represented in the Concert
     },
   },
   "metadata": {
-    "concertinoVersion": "0.1.0-alpha.3",
+    "concertinoVersion": "1.0.0-alpha.7",
     "models": {
       "[email protected]": {
         "concertoVersion": "1.0.0",
@@ -194,23 +179,15 @@ Below is an example of how a simple Concerto model is represented in the Concert
   }
 }
 ```
-```
-
-This example demonstrates how Concertino represents:
-
-1. Fully qualified type names as object keys
-2. Denormalized properties with their constraints
-3. Flattened declarations outside of their model nesting
-4. Metadata including versioning information
-5. Decorators at the model, concept, and property level
-6. Vocabulary with labels and additional terms
 
 The equivalent Concerto CTO file would be:
 
-```cto
+```cs
 @license("Apache-2.0")
 namespace org.example.models@1.0.0
 
+scalar Email extends String regex=/^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/
+
 @Term("Individual")
 @Term_plural("People")
 concept Person {
@@ -220,10 +197,10 @@ concept Person {
   @Term("Family Name")
   o String lastName
 
-  o Integer age optional range=[0, 120]
+  o Integer age optional range=[0, ]
 
   @sensitive
-  o String email optional regex=/^[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}$/
+  o Email email optional 
 
   @Term("Mailing Address")
   o Address address optional

packages/concertino/package-lock.json

@@ -1,6 +1,6 @@
 {
   "name": "@accordproject/concertino",
-  "version": "1.0.0-alpha.3",
+  "version": "1.0.0-alpha.7",
   "description": "Object-centric serialization of Concerto models",
   "homepage": "https://github.com/accordproject/concerto",
   "engines": {
@@ -11,13 +11,14 @@
   "typings": "dist/index.d.ts",
   "scripts": {
     "clean": "rimraf dist",
-    "prebuild": "npm-run-all clean",
+    "prebuild": "npm-run-all clean generate-types",
     "build": "tsc -p tsconfig.build.json",
+    "postbuild": "cp -R src/spec/*.json dist/spec/",
     "pretest": "#npm-run-all lint",
     "lint": "eslint .",
     "test": "vitest run",
     "test:watch": "vitest",
-    "generate-types": "ENABLE_MAP_TYPE=true node --experimental-modules scripts/generateTypes.js"
+    "generate-types": "ENABLE_MAP_TYPE=true node scripts/generateTypes.js"
   },
   "repository": {
     "type": "git",
@@ -31,21 +32,20 @@
   ],
   "author": "accordproject.org",
   "license": "Apache-2.0",
-  "type": "module",
   "dependencies": {
     "@accordproject/concerto-core": "3.22.0",
-    "@accordproject/concerto-types": "3.22.0",
+    "ajv": "^8.17.1",
     "semver": "7.6.3"
   },
   "devDependencies": {
+    "@accordproject/concerto-types": "3.22.0",
     "@accordproject/concerto-codegen": "^3.22.0",
     "@accordproject/concerto-cto": "3.22.0",
     "@accordproject/concerto-util": "^3.22.0",
     "@types/semver": "7.5.8",
     "@typescript-eslint/eslint-plugin": "8.16.0",
     "@typescript-eslint/parser": "8.16.0",
     "@vitest/coverage-v8": "^3.2.4",
-    "ajv": "^8.17.1",
     "eslint": "8.57.1",
     "npm-run-all": "4.1.5",
     "typescript": "^5.7.2",

packages/concertino/package.json

@@ -1,3 +1,4 @@
+/* eslint-disable @typescript-eslint/no-require-imports */
 /*
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -12,16 +13,13 @@
  * limitations under the License.
  */
 
-import fs from 'fs';
-import path from 'path';
-import { ModelManager, ModelUtil } from '@accordproject/concerto-core';
-import { FileWriter } from '@accordproject/concerto-util';
-import { CodeGen } from '@accordproject/concerto-codegen';
-import { fileURLToPath } from 'url';
+'use strict';
 
-// Get the directory name of the current module
-const __filename = fileURLToPath(import.meta.url);
-const __dirname = path.dirname(__filename);
+const fs = require('fs');
+const path = require('path');
+const { ModelManager, ModelUtil } = require('@accordproject/concerto-core');
+const { FileWriter } = require('@accordproject/concerto-util');
+const { CodeGen } = require('@accordproject/concerto-codegen');
 
 const OVERRIDE_DECORATOR = 'CodeGen_TypeScript_Override';