Skip to content

feat(extract): give .svelte and .astro a real AST pass via script masking - #2731

Open
RepairYourTech wants to merge 1 commit into
Graphify-Labs:v8from
RepairYourTech:feat/svelte-astro-script-masking
Open

feat(extract): give .svelte and .astro a real AST pass via script masking#2731
RepairYourTech wants to merge 1 commit into
Graphify-Labs:v8from
RepairYourTech:feat/svelte-astro-script-masking

Conversation

@RepairYourTech

Copy link
Copy Markdown

feat(extract): give .svelte and .astro a real AST pass via script masking

The problem

extract_svelte and extract_astro feed the raw file to the JS grammar:

result = _extract_generic(path, _JS_CONFIG)   # path is the whole .svelte/.astro file

HTML markup is not valid JS, so tree-sitter produces a top-level ERROR node, the
AST pass is abandoned, and both extractors fall back to regex-scraping imports.
Their own docstrings acknowledge this:

The JS tree-sitter parser fed the full .svelte file produces a top-level
ERROR node (HTML markup is not valid JS), so import_statement nodes are never
reached and static imports are silently dropped (#713).

Imports are rescued by regex, so the gap is invisible in import-shaped tests —
but every function, const, interface and type in those files is missing from
the graph
. On a Svelte/Astro codebase that is most of the application.

The fix

graphify already solves exactly this for .vue. _vue_mask_non_script blanks
the markup (preserving \r/\n so line numbers stay accurate) and hands the
result to _extract_generic via source_override — a parameter that exists for
this purpose:

source_override parses the given bytes instead of reading path […] Lets
container formats (e.g. Vue SFCs) mask the wrapper and parse just the embedded
<script>.

Svelte and Astro were simply never wired to it.

  • .svelte reuses _vue_mask_non_script verbatim — the <script> shape
    matches Vue's, including lang= grammar selection (tsx / js / ts).
  • .astro gains _astro_mask, covering the --- fenced frontmatter plus
    client <script> bodies. _astro_best_mask falls back to frontmatter-only for
    the few files whose client script does not concatenate onto the frontmatter as
    valid TS, so those parse clean rather than degrading.
  • _parse_js_tree masks both formats too, so import resolution sees the
    script rather than the markup.

The existing regex rescues are left in place — they still cover template-layer
dynamic imports such as {#await import('./X.svelte')}, which the AST cannot
see, and they dedupe against the AST nodes via existing_ids.

Results

Measured on a Svelte 5 + Astro monorepo (RepairYour.Tech, 1,367 files):

Files Nodes before → after Edges before → after
.svelte 260 931 → 4,307 (4.6×) 1,047 → 6,225
.astro 1,107 2,946 → 6,607 (2.2×) 9,963 → 15,337

Parse cleanliness after masking: 258/258 Svelte, 1,104/1,107 Astro (from zero
clean before — every file errored at line 1).

Regression safety

  • Node/edge digests over ts, tsx, rust, python, sql and json samples
    are byte-identical before and after. Only the .svelte/.astro dispatch
    paths and two new _parse_js_tree branches are touched; the .vue path is
    untouched.
  • test_vue_extraction.py, test_astro_extraction.py and
    test_astro_import_ids.py all pass.
  • Broader extraction sweep: 723 → 731 passing, with the same 231 pre-existing
    failures in my environment (missing optional tree-sitter grammars — identical
    set on a clean checkout).

Tests added

tests/test_svelte_astro_masking.py (8 tests), mirroring
test_vue_extraction.py:

  • masks preserve line counts and blank markup/style
  • Astro frontmatter-only vs. frontmatter+<script> variants
  • a .astro file with no frontmatter masks safely to blank
  • extract_svelte / extract_astro recover declarations and keep imports
  • the masked pass strictly adds nodes over the unmasked baseline

Note for maintainers

This has no issue number yet — happy to open one first if you'd prefer, and to
add the (#NNNN, thanks @…) reference to the CHANGELOG entry to match the
existing convention.

…king

extract_svelte and extract_astro fed the raw file to the JS grammar. HTML
markup is not valid JS, so tree-sitter produced a top-level ERROR node, the
AST pass was abandoned, and both extractors fell back to regex-scraping
imports. Every function, const, interface and type in those files was
invisible to the graph.

graphify already solves this for .vue: _vue_mask_non_script blanks the markup
(preserving newlines so line numbers stay accurate) and hands the result to
_extract_generic via source_override — a parameter whose docstring states it
exists so "container formats (e.g. Vue SFCs) mask the wrapper and parse just
the embedded <script>". Svelte and Astro were never wired to it.

- .svelte reuses _vue_mask_non_script verbatim; the <script> shape matches
  Vue's, including the lang= grammar selection (tsx/js/ts).
- .astro gains _astro_mask, covering the `---` fenced frontmatter plus client
  <script> bodies. _astro_best_mask falls back to frontmatter-only for the
  few files whose client script does not concatenate onto frontmatter as
  valid TS, so those parse clean instead of degrading.
- _parse_js_tree masks both formats too, so import resolution sees the script.

Measured on a Svelte 5 + Astro monorepo (1,367 files):
  .svelte    260 files    931 -> 4,307 nodes    1,047 -> 6,225 edges
  .astro   1,107 files  2,946 -> 6,607 nodes    9,963 -> 15,337 edges

Non-Svelte/Astro extraction is byte-identical: node/edge digests over ts,
tsx, rust, python, sql and json samples are unchanged. The existing
test_vue_extraction, test_astro_extraction and test_astro_import_ids suites
pass, and the broader extraction sweep goes 723 -> 731 passing with the same
231 pre-existing environment failures.

@graphify-labs graphify-labs Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Formal verification. No changes could be formally verified in this run.


Graphify review — findings

This pull request modifies the Svelte and Astro extractors so that both mask out the non-code regions of the file (markup/template/style) and parse the remaining script with the TypeScript grammar, rather than feeding the raw file to the JS grammar. It adds new _astro_mask and _astro_best_mask helpers in resolution.py, wires the masking into extract_svelte, extract_astro, and _parse_js_tree, and keeps the existing regex-based import fallback. A new test file exercises the masking helpers and end-to-end extraction for both file types, and a changelog entry describes the change.

No blocking issues surfaced. 2 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1839 functions depend on the 529 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: _extract_generic() — 20 callers, 23 callees
  • worse: extract_astro() — 8 callers, 6 callees
  • worse: extract_svelte() — 5 callers, 6 callees

Verification — 1839 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 1699 function(s) in the blast radius were not formally verified this run

Formal verification

Could not verify: Could not verify extract\_astro.

The verifier did not have enough to check extract\_astro, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify extract\_svelte.

The verifier did not have enough to check extract\_svelte, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_parse\_js\_tree.

The verifier did not have enough to check \_parse\_js\_tree, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

· 3 more finding(s) on lines outside this diff (see the check run).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant