Skip to content

BEAM-native JS engine and compiler#5

Open
dannote wants to merge 831 commits intomasterfrom
beam-vm-interpreter
Open

BEAM-native JS engine and compiler#5
dannote wants to merge 831 commits intomasterfrom
beam-vm-interpreter

Conversation

@dannote
Copy link
Copy Markdown
Member

@dannote dannote commented Apr 15, 2026

Adds a second QuickJS execution backend that runs decoded QuickJS bytecode on the BEAM, plus an experimental compiler path that lowers QuickJS bytecode to BEAM modules.

Public API and runtime modes

  • mode: :beam support for running JavaScript through the BEAM VM backend
  • mode: :auto compiler-backed execution with fallback
  • strict compiler audit mode via mode: :beam_compiler
  • QUICKBEAM_MODE=beam_compiler support for audit runs
  • raw BEAM disassembly for the :beam backend via QuickBEAM.disasm/2
  • runtime/context integration, handler dispatch, module loading, dynamic import, globals, and interop for the VM path
  • stack traces, source positions, and Error.captureStackTrace

Bytecode VM

  • QuickJS bytecode decoder in Elixir
  • BEAM interpreter for QuickJS bytecode
  • control-flow, stack-depth, type, and diagnostics analysis passes
  • local/argument/capture slot handling, including large-frame tuple mode
  • generator and async-generator execution support
  • promise scheduling and continuation state
  • runtime invocation helpers for compiled and interpreted functions

Compiler backend

  • hybrid compiler from QuickJS bytecode to BEAM modules
  • block lowering, CFG analysis, stack-depth inference, type inference, and optimizer passes
  • compiled runtime helpers for arithmetic, coercion, calls, construction, refs, globals, with-scope, iterators, objects, classes, promises, generators, and errors
  • strict compiler-vs-interpreter audit harnesses and semantic corpus benches
  • compiler opcode coverage reporting
  • compiler perf/ROI benchmark

JavaScript semantics covered

  • Object, Array, Function, String, Number, Boolean
  • Math, JSON, Date, RegExp
  • Map, Set, WeakMap, WeakSet, collection iterators, and self-iterability
  • Symbol, well-known symbols, property keys, and Symbol.toStringTag
  • Promise, async / await, generators, delegated yield*, and async generators
  • Proxy and Reflect, including descriptor/prototype/extensibility invariants
  • TypedArray, ArrayBuffer, BigInt
  • classes, inheritance, super, private fields, private methods, private accessors, static private members, brand checks, and constructor return semantics
  • descriptors, Object.create, Object.assign, Object.fromEntries, freeze/seal/preventExtensions/isExtensible/isFrozen/isSealed
  • array expando properties, sparse arrays, callbacks, iterators, destructuring, spread, rest/default parameters, and arguments basics
  • eval, with, refs, globals, callback/global write freshness, delete semantics, in, instanceof, new, update operators, arithmetic/coercion edge cases, and BigInt operations
  • try/catch/finally, QuickJS gosub/ret finally control flow, nested catches inside finally, break/continue through finally, and catch-region slot preservation
  • for-in enumeration mutation and nullish object coercion for with/destructuring
  • Web API and Node-compat runtime integration used by the VM path

Static analysis and cleanup

  • updated ex_dna, ex_slop, and credo
  • mix lint now runs ExDNA with a zero-clone budget
  • clone debt removed across VM runtime helpers, parser helpers, Web API modules, and collection/object helpers
  • Bandit clock application startup fixed for --no-start test runs
  • shared runtime/context message handling fixed for context state without :mode

Audit and benchmark tooling

  • bench/vm_compiler_compat.exs
  • bench/vm_compiler_corpus.exs
  • bench/vm_compiler_opcode_coverage.exs
  • bench/vm_compiler_perf.exs
  • bench/vm_compiler_semantic_gaps.exs
  • bench/vm_compiler_test262.exs
  • test/support/vm_compiler_audit.ex
  • compiler acceptance and auto-mode tests

Current compiler audit status

  • QuickJS bytecode opcode coverage: 245 / 246 unique opcodes
    • remaining missing opcode: invalid, intentionally unsupported
  • strict compiler Test262 statement window: 1582 / 1582 passing
  • semantic compiler corpus: 1000 / 1000 compiled with zero mismatches, fallbacks, or crashes
  • compiler invoke perf improved from 45.239µs to 42.845µs
  • current compiler perf audit average speedup: 1.113x

Recent validation

  • mix compile --warnings-as-errors
  • mix lint
  • mix dialyzer
  • mix test test/js/parser test/vm/auto_mode_test.exs test/vm/compiler_differential_test.exs test/web_apis/beam_fetch_test.exs test/web_apis/beam_text_encoding_test.exs test/web_apis/beam_buffer_test.exs test/web_apis/beam_streams_test.exs test/web_apis/event_source_test.exs
  • mix test --no-start --exclude napi_addon --exclude napi_sqlite test/web_apis/event_source_test.exs test/web_apis/beam_event_source_test.exs
  • PARSER_BENCH=vm_compiler_semantics ./autoresearch.sh
  • TEST262_LIMIT=1500 TEST262_CASE_TIMEOUT=5000 PARSER_BENCH=vm_compiler_test262 ./autoresearch.sh

Previous broad validation:

  • QUICKBEAM_BUILD=1 MIX_ENV=test mix test
  • MIX_ENV=test QUICKBEAM_BUILD=1 mix test test/vm/js_engine_test.exs --include js_engine --seed 0
  • mix format --check-formatted
  • mix credo --strict
  • mix dialyzer
  • mix ex_dna
  • zlint lib/quickbeam/*.zig lib/quickbeam/napi/*.zig
  • bunx oxlint -c oxlint.json --type-aware --type-check priv/ts/
  • bunx jscpd lib/quickbeam/*.zig priv/ts/*.ts --min-tokens 50 --threshold 0

@dannote dannote force-pushed the beam-vm-interpreter branch from 0eb3475 to 7c1c574 Compare April 15, 2026 14:06
@dannote dannote changed the title BEAM-native JS interpreter (Phase 0-1) BEAM-native JS interpreter Apr 16, 2026
@dannote dannote marked this pull request as ready for review April 16, 2026 08:41
@dannote dannote force-pushed the beam-vm-interpreter branch 2 times, most recently from 75fdba5 to 527d5b9 Compare April 20, 2026 08:45
@dannote dannote changed the title BEAM-native JS interpreter BEAM-native JS engine and compiler Apr 21, 2026
dannote added 23 commits April 25, 2026 04:07
…82 passing, 912 skipped (NIF failures).

Result: {"status":"keep","failures":0,"passing_tests":1582,"skipped_tests":912}
…chieved — all 72 baseline failures fixed across sessions.

Result: {"status":"keep","failures":0,"passing_tests":1582,"skipped_tests":912}
- Fix unused variable warnings (ref → _ref, pc → _pc)
- Move for_of_start_iter helper after catch-all run/6 to fix clause grouping warning
- Remove consecutive blank lines (17 credo issues)
- Add Heap.Caches wrappers for: get/put_fn_atoms, get/put_capture_keys,
  get/put_wrap_cache, get/put_array_proto, get/put_func_proto,
  get/put_builtin_names, get/put_regexp_result, get/put_string_codepoints
- Add Heap delegates for all new Caches functions
- Update Heap.Registry @moduledoc to document every :qb_* key pattern
- Replace raw Process.get/put calls in: interpreter.ex, runtime_helpers.ex,
  interpreter/setup.ex, compiler/runner.ex, compiler/lowering.ex,
  runtime/globals.ex, runtime/object.ex, runtime/regexp.ex, runtime/string.ex,
  object_model/get.ex, object_model/private.ex
- Use Functions.put_home_object/current_home_object in interpreter.ex and private.ex
- lib/quickbeam/vm/compiler/diagnostics.ex: check/1, explain/1, helper_call_counts/1
- bench/compiler_vs_interpreter.exs: compiler vs interpreter on six JS patterns
- Complete lowering tuple refactor (list→tuple for O(1) instruction access)
- Apply analysis tuple refactor to cfg.ex, stack.ex, types.ex
- Fix bench/compiler_vs_interpreter.exs Heap.reset issue
- Targeted benchmarks show compiler ~= interpreter on all patterns
- String concat: 8.42µs (compiler), 9.17µs (interpreter)
- Numeric loop: 61.33µs, 59.17µs
- Array loop: 201.83µs, 207.50µs
- Object field: 615.29µs, 614.04µs
- Function call: 1668.50µs, 1687.06µs
- Closure: 2385.31µs, 2408.00µs
…teger types

Compiler now emits direct BEAM arithmetic/bitwise ops when type analysis
proves operands are integers, bypassing Values.* runtime dispatch.

Specialized: mod→rem, band→band, bor→bor, bxor→bxor, shl→bsl, sar→bsr.
Variables, Objects, Functions, Iterators, Coercion — RuntimeHelpers
is now a thin defdelegate facade. Public API unchanged.
dannote added 30 commits May 4, 2026 13:21
… float constants, remove redundant global whitelist
…s, pass this to iterator.next(), fix declarations for AssignmentPattern/RestElement
… fix check_ctor_return stack handling (5→4)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant