Skip to content

feat(luau): add Luau/Roblox language support (issue #52)#89

Open
CyanoTex wants to merge 4 commits intorepowise-dev:mainfrom
CyanoTex:feat/luau-roblox-support
Open

feat(luau): add Luau/Roblox language support (issue #52)#89
CyanoTex wants to merge 4 commits intorepowise-dev:mainfrom
CyanoTex:feat/luau-roblox-support

Conversation

@CyanoTex
Copy link
Copy Markdown

@CyanoTex CyanoTex commented Apr 20, 2026

Summary

Draft skeleton for issue #52. Adds a luau language tier so Rojo-synced Roblox projects produce a real dependency graph instead of the reported 21 disconnected nodes.

This PR wires the full language path end-to-end with the exception of Rojo default.project.json-aware resolution for game.<Service>... absolute instance paths, which is deliberately scoped to a follow-up.

What's in scope

  • LanguageSpec(tag="luau", ...) in the central registry, replacing the existing git-blame-only lua passthrough spec. One source of truth for both .lua and .luau (Luau's grammar is a strict superset of Lua 5.1).
  • "luau" added to LanguageTag Literal in models.py so the extension flows through FileTraverser/ASTParser instead of being filtered out at models.py:61.
  • LANGUAGE_CONFIGS["luau"] entry in parser.py mapping function_declaration → function and type_definition → type_alias.
  • queries/luau.scm with captures for symbols, require(...) arguments, and call sites.
  • resolvers/luau.py handling:
    • String literals: require("rel/path") — relative then stem fallback.
    • script / script.Parent chains — Roblox semantics where the first Parent is identity (the importer's container directory) and each subsequent Parent walks up.
    • game.<Service>.Path... — registers as external node; full Rojo-tree resolution is the next PR.
  • tree-sitter-luau>=1.2,<2 added to pyproject.toml dependencies.
  • docs/LANGUAGE_SUPPORT.md updated to document the new tier.
  • Unit tests: TestLuauParser in test_parser.py, test_luau_resolver.py for the resolver. Absolute instance-path resolution is pinned with an xfail(strict=True) test documenting the expected end state.

What's deliberately NOT in scope

  • Reading default.project.json to map service subtrees to filesystem directories. The shape is documented in the resolver's module docstring and the failing xfail test. Planned home: core/ingestion/dynamic_hints/rojo.py.
  • Bindings / heritage extractors. Luau modules typically don't use class inheritance; revisit if needed.
  • .lua files in pure-vanilla-Lua projects (not Rojo) will now be AST-parsed where before they were git-blame-only. This is aligned with issue [Feature] Luau/Roblox support: dependency graph parsing and language detection #52's stated goal, but flagging explicitly since it's a behavior change for non-Roblox Lua repos.
  • The pre-existing broken repowise.core.ingestion.parsers reference in pyproject.toml's packages list (unrelated to this PR; surfaced in PR feat: add language support for wiki generation #75 review). Left untouched per the maintainer's preference to keep that fix in its own PR.

Verification

uv run pytest tests/unit/ingestion/ -q
# 227 passed, 1 xfailed
uv run ruff check packages/core/.../luau.py .../registry.py .../parser.py .../models.py .../resolvers/__init__.py tests/.../test_luau*.py tests/.../test_parser.py
# All checks passed

Open questions / risks

  1. tree-sitter-luau ABI pin. The published wheel's [core] extra pins tree-sitter~=0.22, while repowise requires >=0.23,<1. Installing tree-sitter-luau without the [core] extra (which is what the standard dep resolution does) worked locally against tree-sitter 0.23.x — verified by test_parses_require_imports exercising the loader. Worth a CI run against each matrix slot to confirm.
  2. .lua ownership. Taking .lua into the luau spec is a judgment call. Pure-Lua projects will now get AST-parsed; vanilla Lua parses cleanly as a Luau subset. Open to splitting the extensions if preferred.
  3. Resolver for game.<Service>.... Currently returns an external node rather than silently matching the nearest-stem file. This matches the maintainer's "no silent fallback" norm from PRs feat: add OpenRouter as LLM and embedding provider #56 / feat(zai): adopt tier framework for plan-aware rate limiting #83, but means these edges won't land in the graph until the Rojo reader ships.

Test plan

  • uv run pytest tests/unit/ingestion/ -q — green (230 passed, 2 xfailed)
  • uv run ruff check on changed files — clean
  • uv run ruff format --check on changed files — clean
  • Run against a real Rojo project locally via repowise init — done against a private Rojo project + upstream luau-lang/luau (see comment for numbers); Rojo-tree resolution remains pending the follow-up PR
  • Confirm CI ABI compatibility across the matrix

CyanoTex and others added 4 commits April 20, 2026 20:28
Adds a `luau` tier covering `.luau` and `.lua` files: LanguageSpec,
LanguageTag, tree-sitter-luau grammar wiring, .scm captures for
function/type/require, a dedicated import resolver, and unit tests.

The resolver handles `require("rel/path")` and `script` /
`script.Parent` relative instance paths. `game.<Service>...` absolute
instance paths currently register as external nodes; full Rojo-aware
resolution via `default.project.json` is deferred to a follow-up and
pinned by an `xfail` test documenting the expected end state.

Replaces the existing git-blame-only `lua` spec at registry.py rather
than leaving two sources of truth for the same extension set.

Refs: repowise-dev#52

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
parser.py:705 normalizes the captured @import.module text with
.strip("\"'` ") before handing it to resolve_luau_import, but the
resolver's string-literal branch identified literals by checking
arg.startswith('"') — so every require("./path") landed on the
external-node fallback in production.  The unit tests masked this by
passing explicitly-quoted strings, which the production parser never
emits.

Surfaced by running `repowise init --index-only` against the upstream
luau-lang/luau repo (per maintainer's request to test against live
systems): 146/146 string-literal requires — including 107 instances of
require("../bench_support") — went to external when they should have
resolved internally.

Fix:
- Drop the unreachable quote-detection branch.  Detect literal paths
  by elimination after the script./game. regex checks.
- Update the test class to pass *unquoted* input mirroring what the
  parser actually emits.  Add cases for ../ relative paths and stem
  matching so the production handoff is exercised end-to-end.
- Document the parser contract in both the resolver and test module
  docstrings so the next reader doesn't reintroduce the mismatch.

The @alias (.luaurc) branch is deferred alongside the existing Rojo
follow-up and records the reference as external for now.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `script.Parent:WaitForChild("Foo")` and `script.X:FindFirstChild("Foo")`
idioms are how shipping Rojo code loads modules — they guard against the
sync-order race between Rojo's filesystem stream and `require()`.  On
OSRPS they account for **451 of 485** `require(...)` calls (93%);
the bare-dot-chain form that the previous resolver matched accounts for
only 13 (2.7%).  Everything else fell through to the external-node
fallback, leaving the graph's Luau edges almost entirely unresolved.

Fix: normalize `:WaitForChild("Name")` / `:FindFirstChild("Name")` into
`.Name` before the `_SCRIPT_RELATIVE` regex runs.  Both forms have
identical name-resolution semantics (child lookup by string on the
preceding instance), so the path-walking logic in
`_resolve_script_relative` is reused unchanged.

Design notes:
- `WaitForChild`'s optional second `timeoutSeconds` argument is
  discarded by the regex — the timeout is runtime behavior and has no
  bearing on the target module's name.
- External-node labels use the *original* pre-normalization text, so
  the graph shows what was written at the call site rather than a
  rewritten dot-chain form.
- Method chains like `...:WaitForChild("shared"):WaitForChild("Signal")`
  normalize to `.shared.Signal` and resolve through the same code path.

Variable-aliased chains (e.g. `SharedFolder:WaitForChild("X")` where
`SharedFolder = ReplicatedStorage:WaitForChild("OSRPS")`) are a separate
problem — they need lightweight flow tracking to resolve the alias and
are deliberately out of scope here.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Luau's official require-by-string alias mechanism: a `.luaurc` file in
the directory hierarchy declares `{"aliases": {"dep": "./dependency"}}`,
and `require("@dep")` resolves through that map.  Parallel in shape to
tsconfig/jsconfig path aliases (already handled by the TS resolver) and
to the Rojo follow-up this PR already pins.

Counts from `repowise init` against upstream luau-lang/luau: 24 of the
170 string-literal-shaped requires are `@alias` references — every one
currently lands on the external-node fallback.

Implementation needs a `.luaurc` reader layered in via
`core/ingestion/dynamic_hints/luaurc.py`, with:
- parent-directory walk to find the nearest `.luaurc`
- child `.luaurc` overriding parent aliases
- alias values resolved as relative paths from the `.luaurc`'s dir

Out of scope for this PR — recorded here as a strict xfail so the
expected end state is versioned alongside the existing Rojo xfail.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@CyanoTex
Copy link
Copy Markdown
Author

CyanoTex commented Apr 24, 2026

Three follow-up commits pushed on top of 6e5c199 — all in the spirit of test-plan box 4 (running init against real Luau projects). Numbers and rationale below.

Commits

  • fix(luau): string-literal requires never entered the resolverparser.py:705 strips quotes from the captured @import.module text before calling the resolver, but the resolver's string-literal branch keyed off arg.startswith('"'). Result: every require("./path") fell through to the external-node fallback in production. The unit tests passed because they fed explicitly-quoted strings, which the production parser never emits. Fix detects literal paths by elimination (not script.*, not game.*, not @alias) and rewrites the tests to mirror what the parser actually produces.
  • feat(luau): resolve :WaitForChild / :FindFirstChild require chains — On real Rojo code, script.Parent:WaitForChild("Foo") is vastly more common than the bare dot-chain form the original _SCRIPT_RELATIVE regex matched (it's the race-safe idiom against Rojo's filesystem-sync ordering). Normalizes both method-call forms to the dot shape before regex matching; the path-walking logic in _resolve_script_relative is reused unchanged. Variable-aliased chains (SharedFolder:WaitForChild(...)) still go external — that needs lightweight flow tracking and is deliberately out of scope.
  • test(luau): pin .luaurc @alias resolution as a follow-up (xfail) — Second known follow-up alongside the existing Rojo xfail. .luaurc is Luau's official require-by-string alias mechanism (parallel to tsconfig paths). Needs a .luaurc reader; shape is documented in the xfail.

Runtime-verified numbers

Ran repowise init --index-only against two Luau codebases before and after:

Private Rojo project (161 .luau files across src/client, src/server, src/shared):

Before After
Internal (luau→luau) import edges 13 (2.8%) 333 (72.2%)
External edges 448 128

~26× more resolutions. The 128 remaining externals are all known-unsupported shapes: variable-aliased chains (out of scope here), and service-based lookups (ReplicatedStorage:WaitForChild(...), game:GetService(...)) which are in the same family as the existing Rojo follow-up.

Upstream luau-lang/luau (237 Luau files, mostly tests/ and bench/):

Before After
Internal (luau→luau) import edges 1 (0.4%) 113 (45.0%)
External edges 250 138

~113× more resolutions. The 138 remaining break down as:

  • 107× external:script.Parent.bench_support — correct behavior; these are the intentionally-non-matching arm of the require(script.Parent.bench_support) or require("../bench_support") or-chain idiom in bench/*. The ../bench_support branch now resolves (was previously 0 internal, now all 107), so the pattern works end-to-end.
  • 24× @alias references — the .luaurc follow-up (pinned as xfail by the third commit).
  • ~7× scattered test-fixture edge cases.

Suite

uv run pytest tests/unit/ingestion/ -q
# 230 passed, 2 xfailed

Both xfails are the strict-mode documentation-only tests pinning the two deferred follow-ups (Rojo project-tree resolution, .luaurc alias resolution).

One side-finding worth flagging separately

While running init to generate the numbers above, I hit what looked like a serious data-loss bug in packages/cli/src/repowise/cli/mcp_config.py unrelated to this PR — JSON parse errors on an existing ~/.claude/settings.json silently discard the user's config. Filed as #93 with repro and recommended fix, so it's out of this PR's scope and you can triage independently.

-Opus 4.7

@CyanoTex CyanoTex marked this pull request as ready for review April 24, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant