Cross-platform packager: OCI symlink resolution, rewrite-include, rtld_audit removal by wdcui · Pull Request #741 · microsoft/litebox

wdcui · 2026-04-03T14:07:16Z

Summary

Make litebox_packager compile and work on non-Linux hosts (primarily Windows) for the OCI image packaging path.

Remove crate-level #![cfg(target_os = "linux")] gate; gate only ldd-based host mode behind #[cfg(target_os = "linux")]
Add cross-platform file_mode() helper (MetadataExt::mode() on Unix, permission-based heuristic elsewhere)
Cross-platform OCI symlink handling: Track symlinks in-memory (DeferredSymlink) instead of creating OS symlinks (which require special privileges on Windows). Resolve symlink chains through an in-memory map and materialize as file copies or directory placeholders
Add --rewrite-include CLI flag for dlopen'd libraries (e.g., NSS modules) that aren't discovered by the automatic dependency scan
Add OCI platform resolver to always pull linux/amd64 images regardless of host platform
Change Bun-packaged executable detection from a warning to a hard error (these can't be rewritten)
Switch tar format from GNU to USTAR (required by the no_std tar reader)
Normalize \ to / in tar entry paths for Windows compatibility
Delete the no-op build.rs (rtld_audit fully removed from packager)

Bug fixes (not in feature branch)

Whiteout symlink pruning: OCI opaque whiteouts (.wh..wh..opq) and regular whiteouts (.wh.<name>) removed files from disk but didn't prune corresponding entries from the in-memory symlinks vec, causing materialize_symlinks() to resurrect deleted symlinks
Degenerate symlink target resolution: normalize_path() could return an empty PathBuf for targets with excess .. segments, causing resolve_symlink_in_rootfs() to match the rootfs directory itself

Stack

PR Syscall rewriter: interface changes, no_std support, and hardening #738 (merged): Syscall rewriter interface changes
PR Runtime ELF patching, trampoline format changes, and rtld_audit removal #739: Runtime ELF patching + rtld_audit removal
This PR: Cross-platform packager

Test results

219 tests: 204 passed, 15 failed (all TUN/EPERM — no device permissions), 34 skipped
fmt ✅, clippy ✅

sangho2 · 2026-04-07T23:03:25Z

litebox_packager/src/oci.rs

+            std::path::Component::ParentDir => {
+                result.pop();
+            }
+            std::path::Component::CurDir | std::path::Component::RootDir => {}


We might need std::path::Component::Prefix(_) => {} for Windows hosts.

jaybosamiya-ms

I haven't done a deep review of this PR, since it does not seem to require an in-depth review afaict. It is also an overlong PR. Nonetheless, I've added a few inline comments.

Also top-level comment: please simplify the PR description to make sure we don't have overlong descriptions showing up in the squashed history.

Also, maybe a better title is just "Cross-platform packager"?

litebox_packager/src/lib.rs

litebox_packager/src/oci.rs

…olution, rewrite-include flag Make litebox_packager compile and work on non-Linux hosts (primarily Windows) by: - Remove #![cfg(target_os = "linux")] crate-level gate and the dual-main pattern; gate only the host-mode code path behind cfg(target_os) - Add file_mode() helper with unix/non-unix variants to replace MetadataExt::mode() calls - Extract run_host_mode() behind #[cfg(target_os = "linux")] - Track OCI layer symlinks in-memory instead of creating OS symlinks (Windows requires special privileges for symlinks); materialize them after all layers are extracted via resolve_symlink_in_rootfs() - Add is_unix_absolute(), strip_unix_root(), normalize_path() helpers for cross-platform path handling - Force linux/amd64 platform when pulling OCI images - Normalize path separators to Unix-style in tar entries - Add --rewrite-include CLI flag for dlopen'd libraries - Change Bun executable detection from warning to hard error - Switch tar headers from GNU to UStar format

…tion Two bugs found during review: 1. Opaque whiteouts (.wh..wh..opq) and regular whiteouts (.wh.<name>) removed files from disk but did not prune corresponding entries from the in-memory symlinks vec. This caused materialize_symlinks() to resurrect deleted symlinks that a later layer intended to remove. 2. resolve_symlink_in_rootfs() could return Some(rootfs) when a degenerate symlink target with excess .. segments normalized to an empty path via normalize_path(). rootfs.join("") == rootfs, which exists as a directory, causing the entire rootfs to be treated as a resolution target. Guard against empty rel_path at function entry.

- Store Unix permission modes from tar headers in a HashMap during extraction, so permission bits are accurate on non-Unix hosts (Windows) instead of relying on the file_mode() heuristic which returns wrong answers (0o755 for most files). - Build symlink_map once in pull_and_extract and pass through to materialize_symlinks and scan_rootfs (was duplicated in both). - Add lookup_mode() helper that prefers tar header permissions, falls back to file_mode(), defaults to 0o644. - Add existence check for --rewrite-include in finalize_tar (was missing). - Remove redundant --include/--rewrite-include parsing from run_host_mode. - Replace Bun test with rewrite_elf_skips_non_elf_files test. - Add 22 unit tests for normalize_path, is_unix_absolute, strip_unix_root, resolve_symlink_in_rootfs, and lookup_mode. - Add litebox_packager to build_and_test_windows CI job.

… and Windows colon parsing - Normalize all tar entry paths with normalize_path() to prevent path traversal via absolute paths, ../ escape, and ./prefix inconsistency - Normalize hard link source paths for the same protection - Fix root-level opaque whiteout where Path::starts_with("") matches all paths, wiping all in-memory symlinks and permissions - Handle Windows drive letter colons (C:\path) in parse_include so --include/--rewrite-include work correctly on Windows

- Merge --rewrite-include into --include (auto-detect ELF via magic bytes) - Make --include host-mode-only (conflicts_with oci_image) - Remove file_mode() helper; inline MetadataExt::mode() in Linux-only paths - Simplify lookup_mode() to use OCI permissions map only (no host FS fallback) - Add Component::Prefix(_) to normalize_path for Windows path completeness - Remove redundant path.is_absolute() fallback from is_unix_absolute - Consolidate and prune tests in oci.rs - Fix stale UnsupportedBunExecutable match arm from rebase

sangho2 · 2026-04-10T21:41:10Z

litebox_packager/src/lib.rs


    for entry in entries {
-        let mut header = Header::new_gnu();
+        let mut header = Header::new_ustar();


Do we need to use USTAR? It does support up to 255-byte path lengths by default.

sangho2 · 2026-04-10T21:45:55Z

litebox_packager/src/oci.rs

+            // A later layer may override this symlink, so remove any stale
+            // entry with the same rel_path.
+            symlinks.retain(|s| s.rel_path != path);
+            symlinks.push(DeferredSymlink {
+                rel_path: path.clone(),
+                link_target,
+            });


A symlink can be overridden by not only a symlink but also any regular file/directory.

sangho2 · 2026-04-10T21:47:24Z

litebox_packager/src/lib.rs

+                .map(|m| m.mode())
+                .unwrap_or(0o755)
+        };
+        let rewritten = rewrite_elf(&data, &inc.host_path, args.verbose)?;


This ignores --no-rewrite.

sangho2 · 2026-04-10T21:49:19Z

litebox_packager/src/lib.rs

+            use std::os::unix::fs::MetadataExt as _;
+            std::fs::metadata(&inc.host_path)
+                .map(|m| m.mode())
+                .unwrap_or(0o755)


lookup_mode in oci.rs uses 0o644 as a default value.

sangho2 · 2026-04-10T21:50:23Z

litebox_packager/src/oci.rs

+                .into_owned();
+            // A later layer may override this symlink, so remove any stale
+            // entry with the same rel_path.
+            symlinks.retain(|s| s.rel_path != path);


nits. this linearly scans all symlinks.

jaybosamiya-ms

A couple of minor comments.

Also, reminder for earlier top-level comment #741 (review)

jaybosamiya-ms · 2026-04-10T23:57:30Z

litebox_packager/src/oci.rs

+            // Always pull linux/amd64 images regardless of host platform.
+            platform_resolver: Some(Box::new(|entries| {
+                entries
+                    .iter()
+                    .find(|entry| {
+                        entry.platform.as_ref().is_some_and(|p| {
+                            p.os == "linux".into() && p.architecture == "amd64".into()
+                        })
+                    })
+                    .map(|e| e.digest.clone())
+            })),


Shouldn't you pull linux image, but the arch should match the actual host you are running on? Otherwise, running this on an M-series MacBook might be quite surprising, no?

fyi. this commit does a similar thing: 4e1eee7

jaybosamiya-ms · 2026-04-11T00:01:31Z

litebox_packager/src/oci.rs

-pub fn scan_rootfs(rootfs: &Path, verbose: bool) -> anyhow::Result<RootfsFileMap> {
+/// `permissions` provides Unix permission modes captured from tar headers
+/// during extraction, so permission bits are accurate on non-Unix hosts.
+#[allow(clippy::implicit_hasher)]


Nit: prefer expect over allow

This was referenced Apr 3, 2026

Foundation types: platform traits, event extensions, sync changes, AnyMap clone #743

Closed

File descriptor layer: dup/close hooks, fork support, WeakEntryHandle #744

Closed

wdcui requested a review from sangho2 April 3, 2026 21:12

wdcui marked this pull request as ready for review April 3, 2026 21:12

wdcui force-pushed the wdcui/stacked/pr1b-trampoline-format branch from 35d845d to 434eb38 Compare April 5, 2026 04:14

wdcui force-pushed the wdcui/stacked/pr2-packager-crossplatform branch from 7f33b78 to aa7e539 Compare April 5, 2026 04:35

sangho2 reviewed Apr 7, 2026

View reviewed changes

jaybosamiya-ms reviewed Apr 8, 2026

View reviewed changes

wdcui force-pushed the wdcui/stacked/pr1b-trampoline-format branch 2 times, most recently from 15eb87e to 8ea3756 Compare April 8, 2026 18:12

wdcui added 6 commits April 8, 2026 18:40

Remove no-op build.rs from litebox_packager (rtld_audit fully removed)

5fbdeab

wdcui force-pushed the wdcui/stacked/pr2-packager-crossplatform branch from aa7e539 to 74b27eb Compare April 8, 2026 20:00

sangho2 reviewed Apr 10, 2026

View reviewed changes

jaybosamiya-ms reviewed Apr 11, 2026

View reviewed changes

jaybosamiya-ms added the expmt:shadow-kiln Tag to quickly find the different PRs as part of the "shadow kiln" experiment. label Apr 11, 2026

Conversation

wdcui commented Apr 3, 2026

Summary

Bug fixes (not in feature branch)

Stack

Test results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaybosamiya-ms left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaybosamiya-ms left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants