feat(gravity): generic UMAP + HDBSCAN clustering via wasm

[AugustDG]

Adds @bpinternal/gravity — generic density-based clustering (UMAP + HDBSCAN) for high-dimensional embeddings.

The engine is Rust, compiled to WebAssembly with wasm-pack + wasm-bindgen and wrapped in a small TypeScript API (same shape as entities / verel). No native deps, no Python.

import { cluster } from '@bpinternal/gravity'

const { clusters, noise, labels } = cluster({ ids, embeddings, dim }, { minClusterSize: 10 })

• Ships dist/node (CJS), dist/web (ESM), dist/types; the wasm is inlined, so there's no loose .wasm.
• Deterministic, configurable UMAP/HDBSCAN params.
• pnpm test runs the Rust + TS suites. CODEOWNERS: @botpress/orca.

[greptile-apps]

Greptile Summary

This PR introduces @bpinternal/gravity, a new clustering package that compiles a Rust UMAP+HDBSCAN pipeline to wasm32-unknown-unknown and exposes it through a TypeScript cluster() wrapper with no native dependencies. The previous review round's two critical findings (HDBSCAN unwrap() causing WASM traps, and the incorrect cosine-distance formula) have both been addressed.

• Rust engine (rust/): UMAP dimensionality reduction + HDBSCAN density clustering, compiled to WASM; error propagation is now Result-based throughout, with a thread-local error buffer readable by the JS side.
• TypeScript wrapper (src/index.ts): marshals JSON across the WASM boundary, catches non-zero exit codes, and validates the dataset structure before calling into the module.
• CI (.github/workflows/gravity.yml): builds, type-checks, runs both Rust and Vitest test suites, and publishes on master pushes touching gravity/**.

Confidence Score: 5/5

The core pipeline is solid and both prior blocking issues have been resolved cleanly. The only open items are minor CI workflow hygiene points that don't affect correctness or the published artefact.

The HDBSCAN error path is now Result-based with a readable message rather than a WASM trap, the cosine-distance formula correctly normalises both operands, and all five Vitest tests (including the new empty-dataset regression test) pass. The Rust and TypeScript layer boundary is clean: alloc/dealloc round-trips are capacity-correct, the try/finally in TypeScript guarantees dealloc even on trap, and the serde boundary means NaN/Infinity values are caught by deserialisation before reaching any Rust unwrap. The only nits are in the CI workflow configuration.

.github/workflows/gravity.yml — push trigger path filter and unpinned action refs.

Important Files Changed

Filename	Overview
gravity/rust/src/pipeline.rs	Core orchestrator: now returns Result throughout; HDBSCAN error is propagated cleanly instead of panicking. Centroid and cosine-distance calculations use the original (un-normalized) embeddings correctly.
gravity/rust/src/clustering.rs	Cosine-distance formula now divides by both operand magnitudes via l2_norm; sort_by_distances is safe against NaN since l2_norm is clamped to ≥1e-12 and all inputs are finite after serde deserialisation.
gravity/rust/src/wasm/mod.rs	WASM ABI layer: alloc/dealloc pattern is correct (capacity-based round-trip), error and output buffers are thread-local, all error paths call set_error and return non-zero exit code.
gravity/src/index.ts	TypeScript wrapper correctly uses try/finally to guarantee dealloc, reads error buffer on non-zero exit, and caches the WebAssembly.Module (not instance) to avoid recompilation cost.
.github/workflows/gravity.yml	CI builds, tests, and conditionally publishes; push trigger omits the workflow file from its path filter, and the publish action uses a mutable @master ref instead of a pinned SHA.
gravity/src/index.test.ts	Five Vitest tests covering blob recovery, geometry, determinism, option overrides, and the new empty-dataset clean-error path.

_{Reviews (3): Last reviewed commit: "fix(gravity): true cosine distance for c..." | Re-trigger Greptile}

[AugustDG]

@greptile review

[franklevasseur]

My brodeur 🔥

For all other Rust WASM packages that we maintain, we use wasm-pack. I'm not sure if it's worth it, but just so you know, this is the one we usually use.

Also, you should add an index.code-workspace like other packages in the repo so we can open it individually as a VS Code workspace.

{
  "folders": [
    {
      "path": "."
    }
  ],
  "settings": {
    "typescript.tsdk": "node_modules/typescript/lib"
  }
}