Skip to content

feat: Hand-written ASM entry/exit stubs (replace StubEmitter byte arrays) #15

@scc-tw

Description

@scc-tw

Summary

Replace StubEmitter's C++ byte-array stub generation with hand-written, per-architecture ASM entry/exit stubs. These stubs are what the loader patches into the original binary to redirect protected regions to the VM.

Current implementation

StubEmitter (loader/src/X86_64StubEmitter.cpp, ARM64StubEmitter.cpp, X86_32StubEmitter.cpp) generates entry/exit stubs as raw byte vectors in C++:

// Simplified — actual code emits raw bytes
std::vector<uint8_t> X86_64StubEmitter::emit_entry_stub(...) {
    // push rbx; push rbp; push r12-r15; mov rdi, vmctx_ptr; call vm_execute; ...
    return {0x53, 0x55, 0x41, 0x54, ...};
}

This works but is hard to maintain, debug, and extend.

Proposed design

Hand-written ASM stubs per architecture, assembled into object files and linked into the loader:

loader/src/stubs/
    entry_x86_64.S      # SysV AMD64 ABI: save RBX/RBP/R12-R15, setup args, call vm_execute
    exit_x86_64.S       # Restore callee-saved, place return in RAX/XMM0, RET
    entry_arm64.S        # AAPCS64: save X19-X28/FP/LR, setup X0-X7, BL vm_execute
    exit_arm64.S         # Restore, place return in X0/D0, RET via X30
    entry_x86_32.S       # cdecl: save EBX/ESI/EDI/EBP, push args, call vm_execute
    exit_x86_32.S        # Restore, EAX return, RET
    entry_win64.asm      # Win64 ABI: save RBX/RBP/RDI/RSI/R12-R15, shadow space
    exit_win64.asm       # Restore, RAX/XMM0 return

Stub responsibilities

Entry stub (patched at protected region start):

  1. Save all native callee-saved registers
  2. Save native RSP/SP
  3. Load VMContext pointer (from known location)
  4. Transfer native arguments to VM registers (per ABI)
  5. Call vm_execute_with_args()
  6. Place return value in native return register (RAX/X0)
  7. Restore callee-saved registers
  8. RET to original call site

Exit stub (counterpart):

  1. Receive VmExecResult from vm_execute
  2. Extract plaintext return value
  3. Handle multi-value returns (RAX+RDX / X0+X1 for structs)
  4. Handle floating-point returns (XMM0 / D0)
  5. Restore native stack
  6. RET

Benefits over byte-array generation

  • Debuggable: source-level stepping in GDB/LLDB
  • Maintainable: comments, labels, structured code
  • Correct by construction: assembler validates encoding
  • Extensible: easy to add FP register save/restore, CET ENDBR, BTI

Dependencies

Deleted placeholders

The following v1 placeholder files (pure comments, zero code) were deleted:

  • runtime/src/entry_exit/vm_entry_x86_64.S
  • runtime/src/entry_exit/vm_entry_arm64.S
  • runtime/src/entry_exit/vm_exit_x86_64.S
  • runtime/src/entry_exit/vm_exit_arm64.S

Note: these were in runtime/ but entry/exit stubs belong in loader/ (the loader generates them). The new stubs should be in loader/src/stubs/.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions