feat(monorepo): implement orchestrator#731
feat(monorepo): implement orchestrator#731avivkeller wants to merge 3 commits intofeature/monorepofrom
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
65e2c4b to
90b752c
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## feature/monorepo #731 +/- ##
===================================================
Coverage ? 74.37%
===================================================
Files ? 139
Lines ? 13574
Branches ? 1063
===================================================
Hits ? 10096
Misses ? 3473
Partials ? 5 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR restructures the project into a workspace-style monorepo (with packages/core) and replaces the generator registry model with “generator specifiers” that are dynamically loaded and dependency-resolved at runtime.
Changes:
- Convert generator execution to specifier-based loading (
resolveGeneratorGraph) and update threading/config to work with specifiers instead of registry keys. - Move core implementation into
packages/core(new CLI, logger, parsers, utilities, generators) and update workflows/docs/scripts accordingly. - Add/expand generator implementations and supporting utilities (web bundling, sitemap, llms.txt, legacy outputs, etc.) with new tests/READMEs.
Reviewed changes
Copilot reviewed 51 out of 54 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/generators/web/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/sitemap/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/orama-db/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/metadata/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/man-page/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/llms-txt/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/legacy-json/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/legacy-json-all/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/legacy-html/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/legacy-html-all/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/jsx-ast/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/json-simple/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/index.mjs | Removed generator registry (replaced by loader + specifiers). |
| src/generators/ast/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/ast-js/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/api-links/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/addon-verify/index.mjs | Removed old lazy-registry generator entry (migrated to packages/core). |
| src/generators/tests/index.test.mjs | Removed registry-based generator metadata tests. |
| scripts/vercel-build.sh | Updated targets to specifier form (but CLI path still repo-root; see comment). |
| packages/core/src/utils/url.mjs | New URL/file loading + dynamic import helpers + URL-like relative. |
| packages/core/src/utils/unist.mjs | New UNIST stringification utilities + callIfBefore. |
| packages/core/src/utils/remark.mjs | New unified remark/rehype pipelines (HTML + JSX/recma + Shiki integration). |
| packages/core/src/utils/queries/utils.mjs | New helper to detect typed lists in mdast. |
| packages/core/src/utils/queries/index.mjs | New regex query set + UNIST node type guards. |
| packages/core/src/utils/queries/constants.mjs | New shared regex constant(s). |
| packages/core/src/utils/parser.mjs | New loadFromURL utility (parallel to utils/url.mjs). |
| packages/core/src/utils/misc.mjs | New lazy, isPlainObject, and deepMerge. |
| packages/core/src/utils/html-minifier.mjs | New HTML minification helper wrapper around @swc/html-wasm. |
| packages/core/src/utils/generators.mjs | Removed createLazyGenerator (registry-style lazy loader). |
| packages/core/src/utils/file.mjs | New withExt and writeFile helpers (mkdir -p + write). |
| packages/core/src/utils/configuration/types.d.ts | Relaxed config typing to allow specifier targets + arbitrary generator sections. |
| packages/core/src/utils/configuration/templates.mjs | New URL template constants + populate() helper. |
| packages/core/src/utils/configuration/index.mjs | Config now merges defaults from loaded generator map; getConfig fallback updated. |
| packages/core/src/utils/configuration/tests/index.test.mjs | Updated config tests to use a mock loaded-generators map. |
| packages/core/src/utils/array.mjs | New enforceArray helper. |
| packages/core/src/utils/tests/url.test.mjs | Tests for toParsedURL / loadFromURL. |
| packages/core/src/utils/tests/unist.test.mjs | Tests for unist utilities (but callback behavior not asserted; see comments). |
| packages/core/src/utils/tests/parser.test.mjs | Tests for parser loadFromURL. |
| packages/core/src/utils/tests/generators.test.mjs | Removed createLazyGenerator tests and related imports. |
| packages/core/src/threading/parallel.mjs | Parallel worker now uses generator specifiers + passed-in generator object. |
| packages/core/src/threading/index.mjs | New Piscina pool factory module. |
| packages/core/src/threading/chunk-worker.mjs | Worker now imports generator by specifier and caches module promise per specifier. |
| packages/core/src/threading/tests/parallel.test.mjs | Updated parallel tests to use loadGenerator + specifiers. |
| packages/core/src/streaming.mjs | New async-generator collection + caching utilities. |
| packages/core/src/parsers/types.d.ts | New shared parser types (release entries). |
| packages/core/src/parsers/markdown.mjs | New markdown parsing for changelog + index using loadFromURL. |
| packages/core/src/parsers/json.mjs | New JSON type map parser using loadFromURL. |
| packages/core/src/parsers/tests/markdown.test.mjs | Tests for markdown parsers. |
| packages/core/src/parsers/tests/json.test.mjs | Tests for JSON parser. |
| packages/core/src/logger/utils/time.mjs | New timestamp formatting helper. |
| packages/core/src/logger/utils/colors.mjs | New log level styling helper. |
| packages/core/src/logger/types.d.ts | Logger transport context types. |
| packages/core/src/logger/transports/index.mjs | Transport registry (console + GitHub). |
| packages/core/src/logger/transports/github.mjs | GitHub Actions transport implementation. |
| packages/core/src/logger/transports/console.mjs | Console transport implementation. |
| packages/core/src/logger/index.mjs | Logger factory + default logger instance. |
| packages/core/src/logger/constants.mjs | Numeric log levels + tags + color mapping. |
| packages/core/src/loader.mjs | New generator loader + dependency graph resolver by specifier. |
| packages/core/src/generators/web/utils/generate.mjs | Builds client/server wrapper programs and import declarations. |
| packages/core/src/generators/web/utils/data.mjs | Generates and lazily serializes static data injected into bundles. |
| packages/core/src/generators/web/utils/css.mjs | Rolldown plugin to bundle CSS + CSS Modules and emit aggregated CSS asset. |
| packages/core/src/generators/web/utils/chunks.mjs | Virtual chunk require implementation (missing caching; see comment). |
| packages/core/src/generators/web/ui/types.d.ts | Declares injected globals (__STATIC_DATA__, SERVER, CLIENT). |
| packages/core/src/generators/web/ui/package.json | Marks UI bundle as side-effect-free. |
| packages/core/src/generators/web/ui/index.css | Web UI styles and overrides. |
| packages/core/src/generators/web/ui/hooks/useTheme.mjs | Client theme persistence hook. |
| packages/core/src/generators/web/ui/hooks/useOrama.mjs | Client-side Orama DB initialization + fetch/load. |
| packages/core/src/generators/web/ui/constants.mjs | Exposes injected static data. |
| packages/core/src/generators/web/ui/components/SideBar/index.module.css | Sidebar CSS module. |
| packages/core/src/generators/web/ui/components/SideBar/index.jsx | Sidebar component with version select + nav links. |
| packages/core/src/generators/web/ui/components/SearchBox/index.module.css | Search modal/footer styling. |
| packages/core/src/generators/web/ui/components/SearchBox/index.jsx | Search UI component wiring modal/results/hits. |
| packages/core/src/generators/web/ui/components/NoOp.jsx | No-op theme component default. |
| packages/core/src/generators/web/ui/components/NavBar.jsx | Nav bar component (theme toggle, search, GitHub link). |
| packages/core/src/generators/web/ui/components/MetaBar/index.module.css | MetaBar CSS module. |
| packages/core/src/generators/web/ui/components/MetaBar/index.jsx | MetaBar component with stability badges and “view as” links. |
| packages/core/src/generators/web/ui/components/Layout/index.jsx | Default page layout composition. |
| packages/core/src/generators/web/ui/components/CodeBox.jsx | Code box wrapper with language display names + copy-to-clipboard wiring. |
| packages/core/src/generators/web/types.d.ts | Web generator type definitions. |
| packages/core/src/generators/web/template.html | Web HTML template used for output pages. |
| packages/core/src/generators/web/index.mjs | Web generator entry exporting name/depends/defaults + generate(). |
| packages/core/src/generators/web/constants.mjs | JSX import map + speculation rules constants. |
| packages/core/src/generators/web/README.md | Web generator documentation. |
| packages/core/src/generators/types.d.ts | Updated global generator metadata types for specifiers + looser config. |
| packages/core/src/generators/sitemap/utils/createPageSitemapEntry.mjs | Build sitemap entry URL via templates. |
| packages/core/src/generators/sitemap/types.d.ts | Sitemap types + generator type. |
| packages/core/src/generators/sitemap/template.xml | Sitemap XML template. |
| packages/core/src/generators/sitemap/index.mjs | Sitemap generator entry exporting name/depends/defaults + generate(). |
| packages/core/src/generators/sitemap/entry-template.xml | Sitemap entry XML template. |
| packages/core/src/generators/sitemap/README.md | Sitemap generator documentation. |
| packages/core/src/generators/orama-db/utils/title.mjs | Helper for hierarchical heading titles. |
| packages/core/src/generators/orama-db/utils/tests/title.test.mjs | Tests for hierarchical title building. |
| packages/core/src/generators/orama-db/types.d.ts | Orama DB types + generator type. |
| packages/core/src/generators/orama-db/index.mjs | Orama DB generator entry exporting name/depends + generate(). |
| packages/core/src/generators/orama-db/constants.mjs | Orama schema constants. |
| packages/core/src/generators/orama-db/README.md | Orama DB generator documentation. |
| packages/core/src/generators/metadata/utils/yaml.mjs | YAML extraction/normalization/parsing utilities. |
| packages/core/src/generators/metadata/utils/slugger.mjs | Node-flavored slugger using github-slugger + replacements. |
| packages/core/src/generators/metadata/utils/tests/yaml.test.mjs | Tests for YAML parsing/normalization. |
| packages/core/src/generators/metadata/utils/tests/transformers.mjs | Tests for type→MDN/typeMap link transformations. |
| packages/core/src/generators/metadata/utils/tests/slugger.test.mjs | Tests for slugger + replacements. |
| packages/core/src/generators/metadata/types.d.ts | Metadata entry + YAML + stability + heading types. |
| packages/core/src/generators/metadata/index.mjs | Metadata generator entry exporting name/depends/defaults. |
| packages/core/src/generators/metadata/README.md | Metadata generator documentation. |
| packages/core/src/generators/man-page/types.d.ts | Man page generator types. |
| packages/core/src/generators/man-page/template.1 | Man page template content. |
| packages/core/src/generators/man-page/index.mjs | Man page generator entry exporting name/depends/defaults. |
| packages/core/src/generators/man-page/README.md | Man page generator documentation. |
| packages/core/src/generators/llms-txt/utils/buildApiDocLink.mjs | Builds API doc markdown links + descriptions. |
| packages/core/src/generators/llms-txt/utils/tests/buildApiDocLink.test.mjs | Tests for llms-txt link/description building. |
| packages/core/src/generators/llms-txt/types.d.ts | llms-txt generator types. |
| packages/core/src/generators/llms-txt/template.txt | llms.txt template content. |
| packages/core/src/generators/llms-txt/index.mjs | llms-txt generator entry exporting name/depends/defaults. |
| packages/core/src/generators/llms-txt/README.md | llms-txt generator documentation. |
| packages/core/src/generators/legacy-json/utils/buildHierarchy.mjs | Builds hierarchy of metadata entries by depth. |
| packages/core/src/generators/legacy-json/utils/tests/buildSection.test.mjs | Tests for section promotion logic. |
| packages/core/src/generators/legacy-json/utils/tests/buildHierarchy.test.mjs | Tests for hierarchy building. |
| packages/core/src/generators/legacy-json/index.mjs | legacy-json generator entry exporting name/depends/defaults. |
| packages/core/src/generators/legacy-json/constants.mjs | Constants/regexes used by legacy-json parsing/building. |
| packages/core/src/generators/legacy-json/README.md | legacy-json generator documentation. |
| packages/core/src/generators/legacy-json-all/types.d.ts | legacy-json-all output + generator types. |
| packages/core/src/generators/legacy-json-all/index.mjs | legacy-json-all generator entry exporting name/depends/defaults. |
| packages/core/src/generators/legacy-json-all/README.md | legacy-json-all generator documentation. |
| packages/core/src/generators/legacy-html/utils/tableOfContents.mjs | Legacy HTML ToC generation utilities. |
| packages/core/src/generators/legacy-html/utils/replaceTemplateValues.mjs | Legacy template replacement helper. |
| packages/core/src/generators/legacy-html/utils/buildExtraContent.mjs | Legacy extra content (stability overview slot). |
| packages/core/src/generators/legacy-html/utils/buildDropdowns.mjs | Legacy dropdown builders (versions/nav/toc/edit). |
| packages/core/src/generators/legacy-html/types.d.ts | legacy-html generator types. |
| packages/core/src/generators/legacy-html/index.mjs | legacy-html generator entry exporting name/depends/defaults. |
| packages/core/src/generators/legacy-html/assets/js-flavor-esm.svg | New/relocated legacy HTML asset. |
| packages/core/src/generators/legacy-html/assets/js-flavor-cjs.svg | New/relocated legacy HTML asset. |
| packages/core/src/generators/legacy-html/README.md | legacy-html generator documentation. |
| packages/core/src/generators/legacy-html-all/types.d.ts | legacy-html-all generator types. |
| packages/core/src/generators/legacy-html-all/index.mjs | legacy-html-all generator entry exporting name/depends/defaults. |
| packages/core/src/generators/legacy-html-all/README.md | legacy-html-all generator documentation. |
| packages/core/src/generators/jsx-ast/utils/transformer.mjs | Rehype element transformer (tag mapping, table wrapping, footnote relocation). |
| packages/core/src/generators/jsx-ast/utils/getSortedHeadNodes.mjs | Head node sorting helper. |
| packages/core/src/generators/jsx-ast/utils/tests/buildContent.test.mjs | Tests for heading transformations. |
| packages/core/src/generators/jsx-ast/utils/tests/buildBarProps.test.mjs | Updated tests to pass loaded generator defaults into config. |
| packages/core/src/generators/jsx-ast/types.d.ts | jsx-ast generator types. |
| packages/core/src/generators/jsx-ast/index.mjs | jsx-ast generator entry exporting name/depends/defaults. |
| packages/core/src/generators/jsx-ast/README.md | jsx-ast generator documentation. |
| packages/core/src/generators/json-simple/types.d.ts | json-simple generator types. |
| packages/core/src/generators/json-simple/index.mjs | json-simple generator entry exporting name/depends. |
| packages/core/src/generators/json-simple/README.md | json-simple generator documentation. |
| packages/core/src/generators/ast/types.d.ts | ast generator types. |
| packages/core/src/generators/ast/index.mjs | ast generator entry exporting name. |
| packages/core/src/generators/ast/constants.mjs | AST generator constants. |
| packages/core/src/generators/ast/README.md | ast generator documentation. |
| packages/core/src/generators/ast-js/types.d.ts | ast-js generator types. |
| packages/core/src/generators/ast-js/index.mjs | ast-js generator entry exporting name. |
| packages/core/src/generators/ast-js/README.md | ast-js generator documentation. |
| packages/core/src/generators/api-links/utils/checkIndirectReferences.mjs | Helper for indirect reference resolution in JS AST exports. |
| packages/core/src/generators/api-links/types.d.ts | api-links generator types. |
| packages/core/src/generators/api-links/index.mjs | api-links generator entry exporting name/depends/defaults. |
| packages/core/src/generators/api-links/constants.mjs | api-links constants. |
| packages/core/src/generators/api-links/tests/fixtures/root.js | Test fixture for api-links. |
| packages/core/src/generators/api-links/tests/fixtures/reverse.js | Test fixture for api-links. |
| packages/core/src/generators/api-links/tests/fixtures/prototype.js | Test fixture for api-links. |
| packages/core/src/generators/api-links/tests/fixtures/mod.js | Test fixture for api-links. |
| packages/core/src/generators/api-links/tests/fixtures/exports.js | Test fixture for api-links. |
| packages/core/src/generators/api-links/tests/fixtures/class.js | Test fixture for api-links. |
| packages/core/src/generators/api-links/tests/fixtures/buffer.js | Test fixture for api-links. |
| packages/core/src/generators/api-links/tests/fixtures.test.mjs.snapshot | New snapshot outputs for api-links fixtures. |
| packages/core/src/generators/api-links/tests/fixtures.test.mjs | Updated api-links tests for specifier-based loading + new entry points. |
| packages/core/src/generators/api-links/README.md | api-links generator documentation. |
| packages/core/src/generators/addon-verify/utils/section.mjs | Section parsing helpers for addon-verify. |
| packages/core/src/generators/addon-verify/utils/generateFileList.mjs | Generates build tree files (binding.gyp + rewritten JS). |
| packages/core/src/generators/addon-verify/utils/tests/section.test.mjs | Tests for addon-verify section utilities. |
| packages/core/src/generators/addon-verify/utils/tests/generateFileList.test.mjs | Tests for addon-verify file list generation. |
| packages/core/src/generators/addon-verify/types.d.ts | addon-verify generator types. |
| packages/core/src/generators/addon-verify/index.mjs | addon-verify generator entry exporting name/depends. |
| packages/core/src/generators/addon-verify/constants.mjs | addon-verify constants. |
| packages/core/src/generators/addon-verify/README.md | addon-verify generator documentation. |
| packages/core/shiki.config.mjs | New Shiki configuration for theme/lang subset. |
| packages/core/package.json | New core package manifest with exports for ./generators/*. |
| packages/core/bin/utils.mjs | CLI error wrapper helper. |
| packages/core/bin/commands/index.mjs | CLI command registry. |
| packages/core/bin/commands/generate.mjs | CLI generate command updated for specifiers + graph resolution. |
| packages/core/bin/cli.mjs | New CLI entry point within core package. |
| packages/core/README.md | New package README for packages/core. |
| package.json | Root converted to workspace root (private) and scripts updated to run across packages. |
| eslint.config.mjs | Updated ignore paths + workspace file globs. |
| docs/configuration.md | Updated CLI target example to use generator specifiers. |
| README.md | Updated usage examples to use generator specifiers. |
| .prettierignore | Updated ignore paths for moved templates/fixtures. |
| .github/workflows/generate.yml | Updated generator targets to specifiers + CLI path to packages/core/bin/cli.mjs. |
| .github/workflows/codespell.yml | Updated codespell skip path (but currently incorrect; see comment). |
| .github/workflows/ci.yml | Updated CI trigger config (PR branches filter removed). |
| .c8rc.json | Updated coverage excludes for moved assets/UI directories. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
90b752c to
b6d7436
Compare
1155fbd to
c744c0d
Compare
85192b7 to
7d887fd
Compare
2f0ee75 to
d2bf0c9
Compare
7d887fd to
a1a7f40
Compare
a1a7f40 to
f0b7df4
Compare
d2bf0c9 to
72b3de6
Compare
f0b7df4 to
d098db8
Compare
caaa0f9 to
c7f5125
Compare
|
Description
This PR changes our generation logic to, instead of using a generator registry, import generators as specifiers. At the beginning of a pipeline, only the specifiers needed for generation are loaded. Because of this,
generate.mjsandindex.mjsdon't really even need to be seperate.Validation
Every test and generation pipeline should succeed.
Check List
node --run testand all tests passed.node --run format&node --run lint.