Syscall rewriter: interface changes, no_std support, and hardening by wdcui · Pull Request #738 · microsoft/litebox

wdcui · 2026-04-02T03:35:26Z

Add no_std support to litebox_syscall_rewriter (behind a feature flag, using alloc/BTreeSet). Harden ELF rewriting with input validation (Bun executables, relocatable objects), phdr alignment fixup, and checked arithmetic replacing panicking casts. Change hook_syscalls_in_elf to return skipped unpatchable syscall addresses alongside the rewritten binary, add a new patch_code_segment API for runtime in-place patching, and enrich error variants with context strings. Unpatchable syscalls are now replaced with ICEBP; HLT instead of being silently ignored.

Add skipped_addrs output parameter to hook_syscalls_in_elf for reporting unpatchable syscall locations. Add no_std/alloc support with BTreeSet replacing HashSet for deterministic iteration. Add Bun executable detection (UnsupportedBunExecutable error), ET_REL object file rejection, phdr alignment fixup, fork-to-vfork patching, UD2 replacement for unpatchable syscalls, found_any tracking to move NoSyscallInstructionsFound to inner function, and println! removal. Add patch_code_segment public API for in-place code patching. Add unit tests for new functionality and update snapshot tests with address normalization. Update all callers (runner, optee, packager) to 3-arg API.

Reject unsupported Bun executables during packaging, keep rewritten trailer semantics compatible with the loader, and turn malformed rewrite failures into explicit errors.

Match CI rustfmt output for the x86 trampoline comments in the rewritten ELF path.

sangho2

left some comments.

sangho2 · 2026-04-06T17:13:18Z

litebox_syscall_rewriter/src/lib.rs

+        if off + 5 <= buf.len() && patch_end <= buf.len() && off + 5 <= patch_end {
+            buf[off] = 0xE9; // JMP rel32
+            buf[off + 1..off + 5].copy_from_slice(&rel32.to_le_bytes());
+        } else {


Two potential issues:

Overwriting endbr64 (if fork_file_offset is the starting address of fork)

No NOP-filling

Good catch. Fixed.

sangho2 · 2026-04-06T17:14:36Z

litebox_syscall_rewriter/src/lib.rs

+) -> Option<(u64, u64, i32)> {
+    use object::ObjectSymbol as _;
+
+    let mut fork_vaddr = None;


We might need to patch _Fork as well.

I'm not sure about this one.

Both libc_fork and _Fork eventually invoke the fork syscall. The difference is whether they call pthread_atfork or not (mostly libc or other libraries' internal behaviors). Speaking of pthread_atfork, doing not call this function might result in library internal consistency break. https://docs.openssl.org/3.2/man3/OPENSSL_fork_prepare/ libc_vfork doesn't call atfork, so our rewriter might need to handle this.

I will remove the fork->vfork patching. I think we should just work with the fork syscall (when we get to the support for multi processes and vfork/fork).

litebox_syscall_rewriter/src/lib.rs

sangho2 · 2026-04-06T17:16:41Z

litebox_syscall_rewriter/src/lib.rs

+    // input, not executable code. Rewriting instructions or appending
+    // trampoline data would corrupt the object file for the linker.
+    // Check the ELF e_type field (bytes 16..18) before doing any work.
+    if input_binary.len() >= 18 {


why do we run this rewriter against .o?

This is defense in depth. The broker right now just checks if it's an elf file so it may call the rewriter with a .o file.

litebox_syscall_rewriter/src/lib.rs

jaybosamiya-ms

I have not done a deep dive into reviewing this PR, but I noticed a few things when looking through it. I do think that the rewriter would benefit from a cleanup pass in the future, so for now, I do not want to block PRs to tweak it to get no_std support.

litebox_syscall_rewriter/Cargo.toml

litebox_syscall_rewriter/src/lib.rs

jaybosamiya-ms · 2026-04-07T22:57:42Z

litebox_syscall_rewriter/src/lib.rs

+fn checked_add_u64(base: u64, addend: u64, context: &'static str) -> Result<u64> {
+    base.checked_add(addend)
+        .ok_or_else(|| Error::ParseError(format!("{context} address overflow")))
+}
+
+fn checked_sub_u64(base: u64, subtrahend: u64, context: &'static str) -> Result<u64> {
+    base.checked_sub(subtrahend)
+        .ok_or_else(|| Error::ParseError(format!("{context} address underflow")))
+}
+
+fn rel32_bytes(target: u64, base: u64, context: &'static str) -> Result<[u8; 4]> {
+    let disp = i128::from(target) - i128::from(base);
+    let disp = i32::try_from(disp).map_err(|_| {
+        Error::ParseError(format!(
+            "{context} displacement out of range: target {target:#x}, base {base:#x}"
+        ))
+    })?;
+    Ok(disp.to_le_bytes())
+}


Nit: I'm not fully sure why these are ParseError. These seem to be helpers that indeed do the right thing for the current callers (afaict), but the names imply more general, at which point the errors would be actively annoying.

I don't fully understand your comment here. Did you mean to replace ParseError with something like PatchError?

It is a bit of a nit, it is that checked_sub_u64 (for example) does not indicate anything about it being parsing related, but it doing a ParseError is surprising, that's all.

It is fine to leave it as-is.

litebox_syscall_rewriter/src/lib.rs

…riants, use ICEBP;HLT trap - Change hook_syscalls_in_elf and patch_code_segment to return (Vec<u8>, Vec<u64>) instead of taking &mut Vec<u64> for skipped addresses - Rename UnsupportedBunExecutable to UnsupportedExecutable(String) and add context to UnsupportedObjectFile(String) - Add PatchError variant for address arithmetic failures (previously ParseError) - Replace UD2 (0F 0B) with ICEBP;HLT (F1 F4) for poisoned syscalls to avoid confusion with unreachable!() paths - Make anyhow/clap features use dep: syntax in Cargo.toml - Update all callers in packager, runner, and tests

github-actions · 2026-04-08T04:54:36Z

🤖 SemverChecks 🤖 ⚠️ Potential breaking API changes detected ⚠️

Click for details

--- failure enum_unit_variant_changed_kind: An enum unit variant changed kind ---

Description:
A public enum's exhaustive unit variant has changed to a different kind of enum variant, breaking possible instantiations and patterns.
        ref: https://doc.rust-lang.org/reference/items/enumerations.html
       impl: https://github.com/obi1kenobi/cargo-semver-checks/tree/v0.47.0/src/lints/enum_unit_variant_changed_kind.ron

Failed in:
  variant Error::UnsupportedObjectFile in /home/runner/work/litebox/litebox/litebox_syscall_rewriter/src/lib.rs:38

jaybosamiya-ms · 2026-04-08T19:55:21Z

litebox_syscall_rewriter/src/lib.rs

+/// not trap on Linux in userspace, but `HLT` does (SIGILL in ring 3), and the
+/// `F1` prefix makes it easy for a signal handler to distinguish an
+/// intentional break from a spurious one.


This comment is clearly incorrect.

$ cat x.c int main(void) { __asm__ volatile("hlt"); return 0; } $ gcc -o x x.c -g $ gdb ./x Reading symbols from ./x... (gdb) r Starting program: /tmp/tmp.2026-04-07/15-35-06/polished-lumpsucker/x [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Program received signal SIGSEGV, Segmentation fault. main () at x.c:2 2 __asm__ volatile("hlt");

sangho2 · 2026-04-10T17:40:44Z

litebox_syscall_rewriter/src/lib.rs

+        if p_type == object::elf::PT_PHDR {
+            // PT_PHDR — update p_offset to match new location
+            let p_offset_off = entry_off + 8;
+            let old_p_offset =
+                u64::from_le_bytes(buf[p_offset_off..p_offset_off + 8].try_into().unwrap());
+            if old_p_offset == e_phoff {
+                let new_p_offset = (old_p_offset + padding as u64).to_le_bytes();
+                buf[p_offset_off..p_offset_off + 8].copy_from_slice(&new_p_offset);
+            }
+            // The PHDR segment size should match the phdr table; no change needed.
+        }


currently, this code does not update PT_PHDR.p_vaddr/PT_PHDR.p_paddr.

wdcui force-pushed the wdcui/stacked/pr1c-rewriter-interface branch from b9cfe00 to 913c8b3 Compare April 2, 2026 04:17

wdcui requested review from CvvT and sangho2 April 2, 2026 04:17

wdcui force-pushed the wdcui/stacked/pr1c-rewriter-interface branch 2 times, most recently from dacdaf5 to d377b78 Compare April 2, 2026 04:32

wdcui force-pushed the wdcui/stacked/pr1c-rewriter-interface branch from d377b78 to 3719609 Compare April 2, 2026 04:50

wdcui added 2 commits April 4, 2026 14:47

Syscall rewriter: harden ELF rewrite edge cases

bcb8f98

Reject unsupported Bun executables during packaging, keep rewritten trailer semantics compatible with the loader, and turn malformed rewrite failures into explicit errors.

Fix syscall rewriter formatting

eeca867

Match CI rustfmt output for the x86 trampoline comments in the rewritten ELF path.

sangho2 reviewed Apr 6, 2026

View reviewed changes

wdcui added 2 commits April 7, 2026 01:38

address review comments

5081c53

fixed a bug introduced in the last commit

98ea175

jaybosamiya-ms reviewed Apr 7, 2026

View reviewed changes

wdcui added 2 commits April 7, 2026 21:15

Remove fork-to-vfork patching from syscall rewriter

9fc38e2

wdcui changed the title ~~Syscall rewriter: interface changes, no_std support, and bug fixes~~ Syscall rewriter: interface changes, no_std support, and hardening Apr 8, 2026

jaybosamiya-ms reviewed Apr 8, 2026

View reviewed changes

sangho2 reviewed Apr 10, 2026

View reviewed changes

jaybosamiya-ms added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 11, 2026

Conversation

wdcui commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sangho2 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sangho2 Apr 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jaybosamiya-ms left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Apr 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wdcui commented Apr 2, 2026 •

edited

Loading

sangho2 Apr 8, 2026 •

edited

Loading