Type re-exporting similar to wally-package-types#246
Open
stravant wants to merge 2 commits into
Open
Conversation
* Built a type re-exporting mechanism similar to wally-package-types directly into wally so that exported types from packages can be used by the consuming code. * We cannot directly use the same approach as wally-package-types did for a couple of key reasons: * It relies on a source map to locate where the root module of the package is located. Wally packages do not generally include this source map out of the box. Instead, we will assume that Rojo is being used and extract the location of the root package from the Rojo project file. You could theoretically use Wally with something other than Rojo but this is not done and practice, and even if you did the fail case would simply be not getting re-exported types. * It relies on an additional Luau parsing library dependency to do its work. This would not be an issue if Wally were regularly updated. However Wally is not well maintained, meaning that the type re-exporting code will ideally continue to work without needing future updates. If we used a Luau parsing library, that library would not understand future Luau syntax additions and failed parses for libraries using those new syntax features would break re-exporting. * Instead, we use a loose permissive parse which tries to make as few assumptions as possible while locating the export type statements of interest. The core assumptions are that the Module contains valid Luau code and that additional types of string constant / comment syntax will not be added in the future. With these assuptions, we can strip out any non-code text in the form of comments / string constants, and then do a simple parse for export type statements. * One additional complexity exists in the form of export type statements which depend on a non-exported type. In particular as a default type parameter. E.g.: `export type Foo<T = NotExported> = ...`. To avoid this problem we strip out any non-exported default type parameters. That prevents syntax errors while still allowing the types to remain as usable as possible. Fusion is an example of a package that currently has this issue. Bikeshedding welcome on the what name the MODULE variable in the stub modules should be called.
* Add one integration test which install a dependency that has exported types to process. * Add unit tests to extract_types module. * Have to update insta module and deal with some deprecations from that because tests no longer run on a modern toolchain without the update.
Author
With that I believe the PR is in a ready to merge state once it gets code review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Built a type re-exporting mechanism similar to wally-package-types directly into wally so that exported types from packages can be used by the consuming code. We cannot directly use the same approach as wally-package-types did for a couple of key reasons:
wally-package-types relies on a source map to locate where the root module of the package is located. Wally packages do not generally include this source map out of the box. Instead, we will assume that Rojo is being used and extract the location of the root package from the Rojo project file. You could theoretically use Wally with something other than Rojo but this is not done and practice, and even if you did the fail case would simply be not getting re-exported types.
wally-package-types takes an additional Luau parsing library dependency to do its work. This would not be an issue if Wally were regularly updated. However Wally is not well maintained, meaning that the type re-exporting code will ideally continue to work without needing future updates. If we used a Luau parsing library, that library would not understand future Luau syntax additions and failed parses for libraries using those new syntax features would break re-exporting.
Instead, we use a flat permissive parse which tries to make as few assumptions as possible while locating the export type statements of interest. The core assumptions are that the Module contains valid Luau code and that additional types of string constant / comment syntax will not be added in the future. With these assuptions, we can strip out any non-code text in the form of comments / string constants, and then do a simple parse for export type statements.
One additional complexity exists in the form of export type statements which depend on a non-exported type. In particular as a default type parameter. E.g.:
export type Foo<T = NotExported> = .... To avoid this problem we strip out any non-exported default type parameters. That prevents syntax errors while still allowing the types to remain as usable as possible. Fusion is an example of a package that currently has this issue.Note: I also had to update the
reqwestpackage to a newer version as the current version runs into panics when built on an up to date rust toolchain.Bikeshedding welcome on the what name the MODULE variable in the stub modules should be called.