Merge pull request #425 from AntoineGautier/issue422_routing

Improve routing logic

Author: JayHuLBL

server/bin/install-modelica-dependencies.sh

@@ -1,7 +1,7 @@
 #!/bin/sh
 set -x
 
-MODELICA_BUILDINGS_COMMIT=6263b4788b2013a5a5ac3bf5ef163d453995058c
+MODELICA_BUILDINGS_COMMIT=b399379315641da39b231033b0660100fd6489a5
 MODELICA_JSON_COMMIT=a46a361c3047c0a2b3d1cfc9bc8b0a4ced16006a
 
 parent_path=$( cd "$(dirname "${BASH_SOURCE[0]}")" ; pwd -P )

server/src/parser/README.md

@@ -172,18 +172,21 @@ It creates the following dictionary of modifiers on the option `Buildings.Templa
 In [parser.ts](./parser.ts) `loadPackage` wwill attemp to load the provided path.
 
 ### Template Entry Points
-There is a simple template discovery process with a grep for "__ctrlFlow_template".
+The template discovery process relies on a grep for the following hierarchical class annotations:
 
-The executed command looks as follows:
+- `__ctrlFlow(routing="root")`: Marks the top-level package containing all templates, e.g.,
+`Buildings.Templates`
+- `__ctrlFlow(routing="template")`: Marks individual template classes, e.g.,
+`Buildings.Templates.AirHandlersFans.VAVMultiZone`
 
-```typescript
-const cmd = `grep -rl ${dirPath} -e "${TEMPLATE_IDENTIFIER}"`;
-```
-
-This command ultimately returns a list of files that contain the unique string.
-
-Each of these file paths then get 'loaded' in [parser.ts](./parser.ts) into a `File` instance, which starts the process of consuming modelica JSON and generating various types of parse `Element`s.
+All template classes, and all packages between the root package and the template classes
+are considered as "entry points" and added to the `templateNodes` array of [loader.ts](./loader.ts).
+This means that the class URI ultimately dictates the explorer tree structure,
+starting from the "root" package which must be unique inside a library.
 
+Based on their URIs, the entry points are loaded in [parser.ts](./parser.ts) into
+`File` instances, which starts the process of consuming modelica JSON and generating
+various types of parsed `Element`s.
 
 ### Type Store
 During the process of creating `Element`s, each element gets registered to a typestore using the modelica path.
@@ -215,23 +218,6 @@ Currently data is NOT being pulled in from the Modelica Standard Library. A simp
 It is likely a good idea to try and separate out 'Modifier' like objects that have a modelicaPath vs. those that do not. `redeclare` and `final` only relate to modifiers that have a `modelicaPath`.
 
 ### Template Entry Points
-
-The current approach is a simplistic and not very flexible. A more robust approach has been discussed:
-
-- Use a flag indicating that a package (in our case Buildings.Templates) is to be considered as the "root" for all template URIs, for instance:
-__ctrlFlow(routing="root")
-- For each template class (for instance Buildings.Templates.AirHandlersFans.VAVMultiZone):
-__ctrlFlow(routing="template")
-
-
->The contract for the template developer will then be that the class URI dictates the explorer tree structure, starting from the "root" package (necessarily unique inside a library).
-So for instance the template Buildings.Templates.AirHandlersFans.VAVMultiZone with the above annotation would yield the following tree structure:
->
->AirHandlersFans
->
->└── VAVMultiZone
->
->Without having to add any annotation to the subpackage Buildings.Templates.AirHandlersFans.
-```
-
-To implement this, the grep command can continue to be used (by changing the template identifier), however the process for finding subpackages would need to be tweaked a bit in the parser since they are not explicitly listed from the grep command.
+The UI doesn't currently support multiple subpackages. Therefore, only the
+parent package of each template class is used to generate the tree structure in the UI.
+See the function `_extractSystemTypes` in [template.ts](./template.ts).

server/src/parser/loader.ts

@@ -5,7 +5,13 @@ import { execSync } from "child_process";
 import { typeStore } from "./parser";
 import config from "../../src/config";
 
-export const TEMPLATE_IDENTIFIER = "__ctrlFlow_template";
+// The following arrays are populated with the ***full class names***
+// of templates identified with class annotation __ctrlFlow(routing="template")
+// and all packages containing templates up to the root package
+// identified with class annotation __ctrlFlow(routing="root").
+export const TEMPLATE_LIST: string[] = [];
+export const PACKAGE_LIST: string[] = [];
+
 export let MODELICA_JSON_PATH = [
   `${config.MODELICA_DEPENDENCIES}/template-json/json/`,
 ];
@@ -15,48 +21,143 @@ export function prependToModelicaJsonPath(paths: string[]) {
   MODELICA_JSON_PATH = [...paths, ...MODELICA_JSON_PATH];
 }
 
+// The following regexp are prettyPrint safe (\s*) when used with GNU (not BSD) grep -z
+const TEMPLATE_ROOT =
+  'annotation.*__ctrlFlow.*name":\\s*"routing".*simple_expression":\\s*"\\\\"root\\\\"';
+const TEMPLATE_IDENTIFIER =
+  'annotation.*__ctrlFlow.*name":\\s*"routing".*simple_expression":\\s*"\\\\"template\\\\"';
+
+type TemplateNode = {
+  className: string;
+  description: string;
+  json: Object;
+  isPackage: boolean;
+  parentName: string | null;
+};
+
+// Master list of TemplateNode objects representing all templates and packages.
+// It is intended to be used in the future when the UI can handle nested packages.
+const templateNodes: TemplateNode[] = [];
+
 export function getClassNameFromRelativePath(filePath: string) {
   filePath = filePath.endsWith(".json") ? filePath.slice(0, -5) : filePath;
   return filePath.replace(/\//g, ".");
 }
 
+export function getClassNameFromJson(json: any): string {
+  return (
+    (json.within ? json.within + "." : "") +
+    json.class_definition[0].class_specifier.long_class_specifier.identifier
+  );
+}
+
+/**
+ * Creates a TemplateNode object from a JSON representation of a Modelica class.
+ * @param json - The JSON object containing the Modelica class definition
+ * @returns A TemplateNode object with class information
+ */
+function createTemplateNode(json: any): TemplateNode {
+  const classDefinition = json.class_definition[0];
+  return {
+    className: getClassNameFromJson(json),
+    description:
+      classDefinition.class_specifier.long_class_specifier.description_string,
+    json: json,
+    isPackage: classDefinition.class_prefixes.includes("package"),
+    parentName: json.within ?? null,
+  };
+}
+
+/**
+ * Executes a system grep command to search for a regular expression pattern in files.
+ * @param regExp - The regular expression pattern to search for
+ * @param dirPath - The directory path to search in
+ * @returns Array of matching file paths or null if no matches found
+ */
+function systemGrep(regExp: string, dirPath: string): string[] | null {
+  const cmd = `grep -lrz '${regExp}' ${dirPath}`;
+  try {
+    return execSync(cmd)
+      .toString()
+      .split("\n")
+      .filter((p) => p != "");
+  } catch (error) {
+    return null;
+  }
+}
+
 /**
  * Finds all entry points that contain the template identifier for a given package.
  * - LIMITATION: This function requires that the package uses
  *   [Directory Hierarchy Mapping](https://specification.modelica.org/maint/3.6/packages.html#directory-hierarchy-mapping)
+ * - Currently, only entryPoints, TEMPLATE_LIST and PACKAGE_LIST are used.
+ * - In the future, when the UI can handle nested packages, entryPoints should be removed
+ *   and templateNodes should be used to create the file tree structure.
  * @param packageName - The Modelica class name of the package to search for entry points
  * @returns An array of objects containing the path and parsed JSON for each entry point found
  */
 export function findPackageEntryPoints(
   packageName: string,
-): { path: string; json: Object | undefined }[] {
-  const entryPoints: { path: string; json: Object | undefined }[] = [];
+): { className: string; json: Object | undefined }[] {
+  const entryPoints: { className: string; json: Object | undefined }[] = [];
   MODELICA_JSON_PATH.forEach((dir) => {
-    // We need a top directory to look up for entry points
-    // so we can simply convert the class name to a relative path
-    // without adding any file extension.
+    // We can simply convert the class name to a relative path without adding any file extension
+    // because we only need a top directory to look up for entry points.
     const dirPath = path.resolve(dir, packageName.replace(/\./g, "/"));
     if (fs.existsSync(dirPath)) {
-      const cmd = `grep -rl ${dirPath} -e "${TEMPLATE_IDENTIFIER}"`;
-      const response = execSync(cmd).toString();
-      entryPoints.push(
-        ...response
-          .split("\n")
-          .filter((p) => p != "")
-          .sort((a, b) => (a.includes("package.json") ? -1 : 1))
-          .map((p) => path.relative(dir, p))
-          .map((p) => {
-            const path = getClassNameFromRelativePath(p);
-            return {
-              path: path,
-              json: loader(path),
-            };
-          }),
-      );
+      // Find all template files.
+      const templatePaths = systemGrep(TEMPLATE_IDENTIFIER, dirPath);
+
+      // Populate arrays storing templates.
+      templatePaths?.forEach((p) => {
+        const templateJson = loader(p);
+        const templateNode = createTemplateNode(templateJson);
+        TEMPLATE_LIST.push(templateNode.className);
+        templateNodes.push(templateNode);
+      });
+
+      // Find root package.
+      const rootPackagePath = systemGrep(TEMPLATE_ROOT, dirPath)?.[0];
+      if (!rootPackagePath) {
+        console.error("Error: root package not found in " + dirPath);
+        process.exit(1);
+      }
+      const rootPackageJson = loader(rootPackagePath);
+      const rootPackageName = getClassNameFromJson(rootPackageJson);
+
+      // Iterate over all template files up to the root package and populate
+      // templateNodes, TEMPLATE_LIST, PACKAGE_LIST and entryPoints.
+
+      for (let templateJson of [...templateNodes.map(({json}) => json)]) {
+        let packageName = (templateJson as any).within;
+        while (packageName && packageName !== rootPackageName) {
+          const packagePath = getPathFromClassName(packageName, dir);
+          if (!packagePath) {
+            break;
+          }
+          const packageJson = loader(packagePath);
+          // If the package is already in the templateNodes array, its parents have also been added.
+          if (
+            templateNodes
+              .map(({ className }) => className)
+              .includes(packageName)
+          ) {
+            break;
+          }
+          if (!packageJson) {
+            continue;
+          }
+          const packageNode = createTemplateNode(packageJson);
+          PACKAGE_LIST.push(packageNode.className);
+          templateNodes.push(packageNode);
+
+          packageName = (packageJson as any).within;
+        }
+      }
     }
   });
 
-  return entryPoints;
+  return templateNodes.map(({ className, json }) => { return { className, json }; });
 }
 
 /**