Robustness: safe whole-tree scans, serve-html, resolution logging, O(n^2) + serialize fixes, silent-label fix

[Daz-n]

Summary

Hardens graphify for large / whole-tree scans and headless deployments, fixes a silent labeling failure, and removes an O(n²) plus a redundant serialization cost. All changes are additive and behind existing conventions; determinism of graph.json is preserved.

Motivated by a real graphify extract / run on a server: it took ~2h, ingested ~1.18M references from the Go module cache, showed no progress during a ~90-min post-AST phase, and community labeling silently produced Community N placeholders because the openai package (required by the ollama backend's OpenAI-compatible path) wasn't installed.

Changes (4 commits)

1. feat(scan) — safe whole-tree scans + serve-html

• Default excludes for virtual filesystems (/proc,/sys,/dev,/run) and vendored caches (Go module cache gated on the go/pkg/mod path shape, plus .cargo/.rustup/.m2/.gradle/.pnpm-store), wired into the walk's prune loop. Escapable via GRAPHIFY_NO_DEFAULT_EXCLUDES=1 and ! re-include negation.
• --one-file-system (alias --xdev) stops crossing mount points; auto-enabled with a notice when the root is / or $HOME.
• Loud warning when scanning / / $HOME with no excludes.
• New serve-html command serves graphify-out/ over HTTP (--host/--port/--dir) so headless hosts don't hand-roll python -m http.server; warns on 0.0.0.0.

2. perf(resolve) — observability + perf

• Per-resolver timing logs and a periodic counter in the cross-file call loop (the post-AST phase was previously silent).
• _resolve_csharp_type_references: build a label→stub dict once instead of scanning all nodes per edge — O(cs_edges·all_nodes) → O(n).
• lru_cache on the pure, hot _lang_family / _lang_is_case_insensitive.

3. fix(llm) — no more silent placeholder labels

• Distinct BackendConfigError; a missing backend SDK/key now prints an always-visible, actionable message and a preflight fails fast. The broad except is narrowed to recoverable network/API/JSON errors.
• detect_backend prints a one-time notice when a cloud backend is auto-selected (identifiers/content will be sent there). Selection priority unchanged.

4. perf(export) / feat(viz)

• Replace the per-element json.dumps sort key in to_json (a full extra serialization of every node/edge) with stable identity-field keys; output stays deterministic. Refuse-to-shrink guard untouched.
• GRAPHIFY_VIZ_OFFLINE=1 references a locally vendored vis-network.min.js with graceful CDN fallback; SRI + crossorigin on the CDN tag; actionable >5000-node message and an explicit "aggregated view" notice.

Testing

• python -m py_compile on all 9 touched files: OK.
• Test suites not requiring the (optional, absent) tree_sitter/rapidfuzz deps: 437 passed, 2 skipped (test_detect, test_language_resolvers, test_export, test_serve, test_serve_http).
• Functionally verified against a real graph: serve-html serves graph.html (HTTP 200) and warns on 0.0.0.0; default-exclude logic prunes go/pkg/mod and .cargo while keeping a real src/mod dir; /proc,/sys,/dev,/run skipped; / and $HOME flagged as high roots.

Deliberately deferred (kept out to keep this reviewable)

Streaming/on-disk serialization, incremental/localized clustering, parallelizing the resolution phase (parse-once tree reuse), a WebGL/level-of-detail renderer, and slimming the duplicated skill-*.md set. Happy to split any commit into its own PR if you prefer.

🤖 Generated with Claude Code

[graphify-labs]

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 2 advisory finding(s) below merit a look before merge.

Formal verification. 1 change(s) alter behavior, breaking input(s) attached.

Behavior changes: \_community\_article changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_community\_article behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.

Evidence: On input \{"G":"\(lambda \_g: \(\_g\.add\_nodes\_from\(\[\(1, \{\}\), \(2, \{\}\), \(3, \{\}\)\]\), \_g\.add\_edges\_from\(\[\(1, 2, \{\}\), \(1, 3, \{\}\), \(2, 3, \{\}\)\]\), \_g\)\[\-1\]\)\(\_\_import\_\_\('networkx'\)\.Graph\(\)\)","cid":"''","nodes":"\[1, 2, 3\]","label":"'h\\u00e9llo w\\u00f6rld'","labels":"\{'a': 1, 'b': 2\}","cohesion":"100\.0","node\_community":"\{'a': None\}","resolver":"\{'a':…, the old code produced '\# héllo wörld\\n\\n\> 3 nodes · cohesion 100\.00\\n\\n\#\# Key Concepts\\n\\n\- \*\*1\*\* \(2 connections\)\\n\- \*\*2\*\* \(2 connections\)\\n\- \*\*3\*\* \(2 connections\)\\n\\n\#\# Relationships… but the new code produces '\# héllo wörld\\n\\n\> 3 nodes · cohesion 100\.00\\n\\n\#\# Key Concepts\\n\\n\- \*\*1\*\* \(2 connections\)\\n\- \*\*2\*\* \(2 connections\)\\n\- \*\*3\*\* \(2 connections\)\\n\\n\#\# Relationships…. Paste that input straight into a regression test.

---

Graphify review — findings

This pull request appears to revert or roll back some previously listed 0.9.41 changelog entries, renaming the "0.9.40" section to "unreleased" and removing several fix descriptions. It also reverts the cache re-anchoring logic in _relativize_source_files_in (removing the CWD-relative handling) and a small change in benchmark.py's label handling. Additionally, it introduces new CLI functionality: a serve-html command and new flags (--one-file-system, --exclude) documented in __main__.py, plus helper functions in cli.py (_is_high_scan_root, _warn_high_root_scan) for detecting and warning about high-level scan roots like / or $HOME. The surface area spans changelog, CLI help/argument handling, cache path normalization, benchmark label access, and a broad set of associated tests.

Worth a look

• benchmark label crash when label is None — graphify/benchmark.py:42 · Escalate · high
  • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review
• benchmark label None no longer guarded, crashes on None label — graphify/benchmark.py:42 · Escalate · high
  • agreed by 2 of 2 members but NOT verified (no proof, no reproducing execution) — consensus is not a verdict; needs human review

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 4503 functions depend on the 2972 functions this change touches.

Health — this change adds coupling hotspots:

• worse: detect() — 86 callers, 17 callees
• worse: dispatch_command() — 2 callers, 119 callees
• worse: to_html() — 17 callers, 11 callees
• worse: detect_backend() — 16 callers, 4 callees
• worse: generate_community_labels() — 6 callers, 4 callees
• new: preflight_backend() — 1 callers, 6 callees

Verification — 4503 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 4444 function(s) in the blast radius were not formally verified this run

Formal verification

Behavior changes: \_community\_article changes behavior, here is the input that shows it.

The verifier found a concrete input on which \_community\_article behaves differently before and after the change. If that change is intended, ship it; if not, this is your bug.

Guarantee: This difference was REPRODUCED, the verifier actually ran both versions on that input and saw them disagree. It is real, not an artifact.

Evidence: On input \{"G":"\(lambda \_g: \(\_g\.add\_nodes\_from\(\[\(1, \{\}\), \(2, \{\}\), \(3, \{\}\)\]\), \_g\.add\_edges\_from\(\[\(1, 2, \{\}\), \(1, 3, \{\}\), \(2, 3, \{\}\)\]\), \_g\)\[\-1\]\)\(\_\_import\_\_\('networkx'\)\.Graph\(\)\)","cid":"''","nodes":"\[1, 2, 3\]","label":"'h\\u00e9llo w\\u00f6rld'","labels":"\{'a': 1, 'b': 2\}","cohesion":"100\.0","node\_community":"\{'a': None\}","resolver":"\{'a':…, the old code produced '\# héllo wörld\\n\\n\> 3 nodes · cohesion 100\.00\\n\\n\#\# Key Concepts\\n\\n\- \*\*1\*\* \(2 connections\)\\n\- \*\*2\*\* \(2 connections\)\\n\- \*\*3\*\* \(2 connections\)\\n\\n\#\# Relationships… but the new code produces '\# héllo wörld\\n\\n\> 3 nodes · cohesion 100\.00\\n\\n\#\# Key Concepts\\n\\n\- \*\*1\*\* \(2 connections\)\\n\- \*\*2\*\* \(2 connections\)\\n\- \*\*3\*\* \(2 connections\)\\n\\n\#\# Relationships…. Paste that input straight into a regression test.

No difference found (not proven): No behavior difference found in \_query\_subgraph\_tokens (not a proof).

The verifier ran both versions of \_query\_subgraph\_tokens on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_relativize\_source\_files\_in.

The verifier did not have enough to check \_relativize\_source\_files\_in, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify dispatch\_command.

The verifier did not have enough to check dispatch\_command, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 23 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly SystemExit — names the real obstacle, not a sampling gap)

Could not verify: Could not verify classify\_file.

The verifier did not have enough to check classify\_file, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify detect.

The verifier did not have enough to check detect, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify detect\_incremental.

The verifier did not have enough to check detect\_incremental, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `root` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_is\_ignored.

The verifier did not have enough to check \_is\_ignored, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in to\_json (not a proof).

The verifier ran both versions of to\_json on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

No difference found (not proven): No behavior difference found in to\_html (not a proof).

The verifier ran both versions of to\_html on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify extract.

The verifier did not have enough to check extract, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `cache_root` is annotated `Path | None` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_import\_js.

The verifier did not have enough to check \_import\_js, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 200 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_lang\_family.

The verifier did not have enough to check \_lang\_family, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `source_file` is annotated `object` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_lang\_is\_case\_insensitive.

The verifier did not have enough to check \_lang\_is\_case\_insensitive, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `source_file` is annotated `object` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_resolve\_csharp\_type\_references.

The verifier did not have enough to check \_resolve\_csharp\_type\_references, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 1 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

Could not verify: Could not verify \_csharp\_extra\_walk.

The verifier did not have enough to check \_csharp\_extra\_walk, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 200 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly AttributeError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify \_extract\_generic.

The verifier did not have enough to check \_extract\_generic, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `path` is annotated `Path` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_resolve\_php\_type\_references.

The verifier did not have enough to check \_resolve\_php\_type\_references, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 1 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

No difference found (not proven): No behavior difference found in detect\_backend (not a proof).

The verifier ran both versions of detect\_backend on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify generate\_community\_labels.

The verifier did not have enough to check generate\_community\_labels, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 1 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

No difference found (not proven): No behavior difference found in \_run\_cli (not a proof).

The verifier ran both versions of \_run\_cli on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify run\_language\_resolvers.

The verifier did not have enough to check run\_language\_resolvers, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `paths` is annotated `Sequence` — outside the synthesizable primitive/collection set

No difference found (not proven): No behavior difference found in \_subgraph\_to\_text (not a proof).

The verifier ran both versions of \_subgraph\_to\_text on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.

Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.

Note: An input the sampler did not try could still differ.

Could not verify: Could not verify \_check\_shrink.

The verifier did not have enough to check \_check\_shrink, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `tmp` is annotated `'Path | None'` — outside the synthesizable primitive/collection set

Could not verify: Could not verify \_rebuild\_code.

The verifier did not have enough to check \_rebuild\_code, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: parameter `watch_path` is annotated `Path` — outside the synthesizable primitive/collection set

· 4 grounded finding(s) anchored inline below; 2 more finding(s) on lines outside this diff (see the check run).

[graphify-labs]

⚠️ Health regression — dispatch_command()

fans out to 119 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

[graphify-labs]

⚠️ Health regression — detect()

fans out to 17 callees (efferent coupling); 86 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

[graphify-labs]

⚠️ Health regression — preflight_backend()

fans out to 6 callees (efferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.

[graphify-labs]

⚠️ Health regression — detect_backend()

16 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.