Stuff for API level v2

NotNite · NotNite · commit a833d2488c44 · 2024-10-03T18:10:36.000-04:00
diff --git a/blog/2023-12-06-reintroducing-moonlight.md b/blog/2023-12-06-reintroducing-moonlight.md
@@ -7,6 +7,8 @@ image: /static/img/wordmark.png
 
 A few days ago, we released a new Discord client mod called [moonlight](/). moonlight is the end result of our ideas and experiences from Discord client modding, influenced by several years of working on other client modding projects. We invite developers to try out moonlight and report feedback on what we can improve.
 
+<!-- truncate -->
+
 ## Yet another Discord mod
 
 Tens if not hundreds of Discord desktop client mods have come and gone, so why make another? Simple; the freedom to innovate and experiment. With moonlight, we're free to do what we want and experiment with the ideas that we've had for years without any limitations. Creating moonlight has offered a fresh slate to build our own systems and play with what we like most, and allows us to try out new things with zero consequence.
diff --git a/blog/2024-10-03-moonlight-api-v2.md b/blog/2024-10-03-moonlight-api-v2.md
@@ -0,0 +1,124 @@
+---
+title: moonlight API v2, mappings, and more
+description: New libraries and utilities for a new version of moonlight
+authors: [notnite, cynosphere, adryd]
+---
+
+Been a while, eh? The last post we wrote for moonlight was [when we first introduced it](/blog/2023-12-06-reintroducing-moonlight.md). Sounds like it's time to change that!
+
+<!-- truncate -->
+
+moonlight development was stalled for a while this year, particularly due to lack of motivation. Less people than expected tinkering with moonlight gave us less reason to update it, and so it fell behind with Discord updates, eventually breaking catastrophically for several months. Thanks to [redstonekasi](https://github.com/redstonekasi) for submitting pull requests for fixing a *lot* of things while none of us had the energy to.
+
+About a week ago, [Ari](https://github.com/adryd325) had the idea of making a centralized repository for Discord client mappings. This short conversation would lead to one of the greatest nerdsnipes in moonlight history.
+
+With the new features we're rolling out, this means new concepts of an "API level". Extensions that do not meet the provided API level will not load. If you don't care about the shiny new toys, jump [here](#what-a-version-bump-means-for-you) for more info on that.
+
+## LunAST: AST-based mapping and patching
+
+The concept of centralized mappings started with [LunAST](https://github.com/moonlight-mod/lunast) (Lunar + AST), a library for manipulating Webpack modules with various ESTree-related tools. LunAST enables you to patch and traverse modules using an AST, which is much more flexible than the current text-based patching with RegEx/strings.
+
+Here's the first LunAST patch we ever wrote, as an idea for how it works. This makes it say "balls" when you click on an image preview (very professional, I know):
+
+```ts
+import type { AST } from "@moonlight-mod/types";
+
+moonlight.lunast.register({
+  name: "ImagePreview",
+  find: ".Messages.OPEN_IN_BROWSER",
+  process({ id, ast, lunast, markDirty }) {
+    const getters = lunast.utils.getPropertyGetters(ast);
+    const replacement = lunast.utils
+      .magicAST(`return require("common_react").createElement(
+  "div",
+  {
+    style: {
+      color: "white",
+    },
+  },
+  "balls"
+)`)!;
+    for (const data of Object.values(getters)) {
+      if (!lunast.utils.is.identifier(data.expression)) continue;
+
+      const node = data.scope.getOwnBinding(data.expression.name);
+      if (!node) continue;
+
+      const body = node.path.get<AST.BlockStatement>("body");
+      body.replaceWith(replacement);
+    }
+    markDirty();
+
+    return true;
+  }
+});
+```
+
+This is a very powerful tool, but we aren't sure how well it'll work for extension developers just yet. If you have a particularly complicated patch in your extension codebase, see if LunAST can help you. Some notes, though:
+
+- AST parsing is expensive! Use `find` when possible to filter for modules to parse.
+- AST patching might break other text-based patches. If you encounter any weirdness, let us know.
+- Text-based patching is not going away, and you should not use AST patching everywhere. This is purely a tool for the harder stuff.
+
+## moonmap: dynamic remapping of Webpack modules
+
+[moonmap](https://github.com/moonlight-mod/moonmap) allows you to find a module, create a proper name for it, and create named exports from minified variable names. This feature was originally in LunAST, but we decided to expand on it and bring it into its own library. Here's a snippet from the mappings project (which we'll get into later):
+
+```ts
+const name = "discord/utils/HTTPUtils";
+moonlight.moonmap.register({
+  name,
+  find: '.set("X-Audit-Log-Reason",',
+  process({ id, moonmap }) {
+    moonmap.addModule(id, name);
+
+    moonmap.addExport(name, "HTTP", {
+      type: ModuleExportType.Key,
+      find: "patch"
+    });
+
+    return true;
+  }
+});
+```
+
+You can then just `spacepack.require("discord/utils/HTTPUtils").HTTP`, and it just works! Magic:tm:.
+
+## mappings: client mod agnostic Discord mappings
+
+[mappings](https://github.com/moonlight-mod/mappings) combines moonmap and LunAST into one project to map out the Discord client. This is an idea that was tested in [HH3](/blog/2023/12/06/reintroducing-moonlight#whats-with-hh3), and we believe that it's stable by now.
+
+The biggest feature about the mappings is that they aren't locked into the moonlight patcher system. Any client mod that can implement moonmap and LunAST into their patching system can use the mappings repository with no extra effort. We hope this can save some duplicated effort across the client modding community.
+
+```ts
+import load from "@moonlight-mod/mappings";
+
+load(moonmap, lunast);
+
+// later, after the modules finish initializing
+const Dispatcher = require("discord/Dispatcher").default;
+```
+
+There's still a lot of work to be done for typing the untyped modules, and for adding new modules. We migrated a majority of types in the Common extension into this library, and most of the things in Common have been removed.
+
+## What a version bump means for you
+
+### As a user
+
+All extensions you are currently using (minus the extensions built into moonlight) will stop working. The developers of those extensions will need to update them, and if those extensions are on [the official repository][official-repo], they will need to be resubmitted and reviewed.
+
+### As an extension developer
+
+Your extensions will need to be updated. See [this new page in the documentation](/docs/ext-dev/migrating-api-levels).
+
+### As another client mod developer
+
+All of the libraries mentioned above can be used by your own code now. Have fun!
+
+## The future of moonlight
+
+moonlight, like the [PlayStation 5](https://en.wikipedia.org/wiki/Category:PlayStation_5-only_games), is pointless when there's nothing to install onto it. As with [last time](/blog/2023-12-06-reintroducing-moonlight.md#whats-next-for-moonlight), we encourage developers to try making extensions. Let us know if there's anything we can improve, and submit your extension to [the official repository][official-repo] if you'd like.
+
+We don't consider this a "moonlight 2.0" as much as "moonlight API version 2". There's no groundbreaking rewrite going on here, just some new libraries to play with.
+
+[official-repo]: <https://github.com/moonlight-mod/extensions>
diff --git a/docs/02-ext-dev/02-helpful-exts.md b/docs/02-ext-dev/02-helpful-exts.md
@@ -11,16 +11,6 @@ These aren't extensions, but rather options in the moonlight config:
 - `loggerLevel`: The level to log at. If not set, defaults to `info`.
 - `patchAll`: Set to `true` to wrap every Webpack module in a function. This slows down client starts significantly, but can make debugging easier.
 
-## Common
-
-Common exposes multiple Webpack modules useful for development:
-
-- `common_react`: Discord's version of React (required to be in scope for JSX)
-- `common_flux`: Discord's version of Flux
-- `common_stores`: A proxy around Discord's Flux stores
-  - Access stores as properties, e.g. `require("common_stores").UserStore`
-- `common_http`: A small HTTP client
-
 ## Disable Sentry & No Track
 
 Both of these extensions do not provide any utilities, but prevent your client from sending metrics to Discord, which lessens the risk of moonlight related errors being reported. These should always be enabled. These are not magical tools to prevent account suspension, and you should always consider safety when writing an extension (especially one that makes requests automatically).
diff --git a/docs/02-ext-dev/04-esm-webpack-modules.md b/docs/02-ext-dev/04-esm-webpack-modules.md
@@ -24,3 +24,5 @@ declare module "@moonlight-mod/wp/sampleExtension_someLibrary" {
 ## CJS and ESM interop
 
 When using `export default` or `export something` in a ESM Webpack module like this, you will need to do `require("ext_id").default` or `require("ext_id").something` to access it. You can also use `module.exports`, but there may be issues that arise from it.
+
+Importing the default export of a mapped Webpack module may have issues.
diff --git a/docs/02-ext-dev/05-pitfalls.md b/docs/02-ext-dev/05-pitfalls.md
@@ -70,9 +70,9 @@ const myElement = <span>Hi!</span>;
 const myElement = React.createElement("span", null, "Hi!");
 ```
 
-Because of this, you need to make sure the React variable is in scope. The easiest way to do this is to use the Common core extension:
+Because of this, you need to make sure the React variable is in scope. `react` is automatically populated as a Webpack module name with [mappings](/docs/02-ext-dev/07-mappings.md):
 
 ```tsx
-const React = require("common_react");
+const React = require("react");
 const myElement = <span>Hi!</span>;
 ```
diff --git a/docs/02-ext-dev/06-webpack.md b/docs/02-ext-dev/06-webpack.md
@@ -92,7 +92,7 @@ export const patches: Patch[] = [
 
 Similar to patching, extensions can also insert their own Webpack modules. These can be required like normal modules (which means they can be used inside of patches and other extensions).
 
-To create a Webpack module, export them from your extension's web entrypoint:
+To create a Webpack module, export them from your extension's web entrypoint, or [use an ESM Webpack module](/docs/02-ext-dev/04-esm-webpack-modules.md):
 
 ```ts
 export const webpackModules: Record<string, ExtensionWebpackModule> = {
@@ -124,4 +124,4 @@ export const webpackModules: Record<string, ExtensionWebpackModule> = {
 };
 ```
 
-You can then require this module using the format `${ext.id}_${webpackModule.name}` (e.g. the `react` Webpack module in the `common` extension has the ID `common_react`).
+You can then require this module using the format `${ext.id}_${webpackModule.name}` (e.g. the `stores` Webpack module in the `common` extension has the ID `common_stores`).
diff --git a/docs/02-ext-dev/07-mappings.md b/docs/02-ext-dev/07-mappings.md
@@ -0,0 +1,39 @@
+# Mappings
+
+moonlight uses [mappings](https://github.com/moonlight-mod/mappings) that automatically rename Webpack modules for you.
+
+## Using import statements with mappings
+
+For mapped modules with types, you can `import` from them like normal in a Webpack module:
+
+```ts
+import Dispatcher from "@moonlight-mod/wp/discord/Dispatcher";
+```
+
+## Using require with mappings
+
+For mapped modules without types, you will need to require them manually. Your build system might not understand your `require`, so you may need to use `spacepack.require`:
+
+```ts
+// React has a type, so this is not required, but this is for demonstration
+const React = spacepack.require("react");
+```
+
+## Adding a mapped module to dependencies
+
+You can add a mapped module to your Webpack module dependencies like normal - just remove the `ext` field.
+
+```ts
+export const webpackModules: Record<string, ExtensionWebpackModule> = {
+  something: {
+    dependencies: [
+      {
+        id: "discord/Dispatcher"
+      },
+      {
+        id: "react"
+      }
+    ]
+  }
+};
+```
diff --git a/docs/02-ext-dev/08-devtools.md b/docs/02-ext-dev/08-devtools.md
diff --git a/docs/02-ext-dev/09-migrating-api-levels.md b/docs/02-ext-dev/09-migrating-api-levels.md
@@ -0,0 +1,21 @@
+# Migrating API levels
+
+When a moonlight API level increases, all extensions with a different API level will refuse to load. The current API level is 2.
+
+As with every update to moonlight, the following apply:
+
+- The types package must be updated: `pnpm add @moonlight-mod/types@latest`
+- The `version` field in your manifest must be updated.
+- The `apiLevel` field in your manifest must be set to the latest API level.
+- If your extension is on [the official repository](/docs/02-ext-dev/03-official-repository.md), it must be resubmitted and reviewed.
+
+## 1 -> 2
+
+The concept of an API level was introduced with API level 2. No API level specified is implicitly assumed to be 1.
+
+With the introduction of [mappings](/docs/02-ext-dev/07-mappings.md), a lot of helper Webpack modules from the Common extension are gone, and moved into the mappings repository. Some notable examples:
+
+- `common_flux`: `discord/packages/flux`
+- `common_fluxDispatcher`: `discord/Dispatcher`
+- `common_react`: `react`
+- `common_components`: `discord/components/common/index`
diff --git a/docs/03-dev/01-setup.md b/docs/03-dev/01-setup.md
@@ -35,3 +35,7 @@ moonlight is split into a [pnpm workspace](https://pnpm.io/workspaces). The proj
 ## Build system
 
 moonlight uses [esbuild](https://esbuild.github.io) as its build system (`build.mjs`). This script is responsible for outputting `injector`, `node-preload`, and `web-preload` into `dist`, along with all core extensions.
+
+## Other dependencies
+
+moonlight uses some other packages that are not in the monorepo, like [LunAST](https://github.com/moonlight-mod/lunast), [moonmap](https://github.com/moonlight-mod/moonmap), and [mappings](https://github.com/moonlight-mod/mappings). When testing local forks, try [pnpm link](https://pnpm.io/cli/link).
