From 13847b2e9ff35142f376eed8cbdf643491f00bfe Mon Sep 17 00:00:00 2001 From: Joseph Date: Mon, 1 Jun 2026 13:42:09 +0200 Subject: [PATCH 1/4] docs: bodySizeLimit measures raw HTTP body (#94137) Started from https://github.com/vercel/next.js/discussions/93989 Noticed we even have a test helper to account for overhead: https://github.com/vercel/next.js/blob/fa4c0def823447e75366153690abeda79c7c4ec2/test/e2e/app-dir/actions/account-for-overhead.js#L1-L11 Let's document that the flag controls the raw http body size, overhead + data --- .../05-config/01-next-config-js/serverActions.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/01-app/03-api-reference/05-config/01-next-config-js/serverActions.mdx b/docs/01-app/03-api-reference/05-config/01-next-config-js/serverActions.mdx index 6d1a594816ef..1975bf9b455e 100644 --- a/docs/01-app/03-api-reference/05-config/01-next-config-js/serverActions.mdx +++ b/docs/01-app/03-api-reference/05-config/01-next-config-js/serverActions.mdx @@ -39,6 +39,8 @@ module.exports = { } ``` +The limit applies to the raw HTTP request body, including the bytes that `multipart/form-data` adds for boundaries, part headers, and field metadata. If you expect uploads close to the configured value, leave some room for this overhead. For typical multipart uploads, an additional 10–20 KB is a reasonable rule of thumb. + ## Enabling Server Actions (v13) Server Actions became a stable feature in Next.js 14, and are enabled by default. However, if you are using an earlier version of Next.js, you can enable them by setting `experimental.serverActions` to `true`. From 11c865c1d6b391fcdab4c1c48a4d57e98b77816c Mon Sep 17 00:00:00 2001 From: Tobias Koppers Date: Mon, 1 Jun 2026 14:25:48 +0200 Subject: [PATCH 2/4] Update ctor from 0.10 to 1.0.6 (#94045) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### What? Updates the `ctor` crate dependency from `0.10` to `1.0.6` in the workspace and migrates the two call sites to the new API. ### Why? We've been running into a rust-analyzer issue with `ctor` 0.10 that the newer release looks like it should fix. While bumping, the major version jump also brings in a number of stability and safety improvements (post-libc-init default priority, required `unsafe` marker, declarative re-export form, expanded platform support). ### How? `ctor` 1.x has several breaking changes that affect us: 1. **`#[ctor]` now requires an explicit `unsafe` marker.** Both call sites were updated: - `turbopack/crates/turbo-tasks-macros-tests/tests/trybuild.rs`: `#[ctor::ctor]` → `#[ctor::ctor(unsafe)]`. The function body already wraps `std::env::remove_var` in `unsafe`, so no further change was needed. - `turbopack/crates/turbo-tasks-macros/src/value_impl_macro.rs`: the proc-macro emits a vtable registration ctor for every `#[turbo_tasks::value_impl]`. Migrated to the new declarative form (see next bullet). 2. **`crate_path = ...` is deprecated in favor of `ctor::declarative::ctor!`.** The ctor 1.0.6 docs explicitly recommend the declarative macro for crates that re-export ctor (`turbo-tasks` re-exports it via `macro_helpers::ctor` so user crates don't need a direct dep). The emission in `value_impl_macro.rs` was rewritten from ```rust #[turbo_tasks::macro_helpers::ctor::ctor( crate_path = turbo_tasks::macro_helpers::ctor, )] #[allow(non_snake_case)] fn #vtable_register_ident() { ... } ``` to ```rust turbo_tasks::macro_helpers::ctor::declarative::ctor! { #[ctor(unsafe)] #[allow(non_snake_case)] fn #vtable_register_ident() { ... } } ``` 3. **Default priority changed.** In 0.x the default priority was `0` (run before C library init); in 1.x the default is `default` (≈500, after libc init). We don't specify a priority, and the registration ctors only push into a `Vec` guarded by a registry — they don't touch libc — so the later timing is safe and we accept the new default. The lazy `VTableRegistry::finalize` step (driven by the `VALUES` `LazyLock`) is unchanged, so visible ordering semantics for downstream code are preserved. 4. **Other 1.x breaking changes verified as non-issues for this repo:** - `dtor` was split into its own crate — not used here. - `priority = naked` syntax removed — not used. - MSRV bumps: base is 1.60, only the optional `priority` feature requires 1.85 — we don't enable `priority`. - `used_linker` / `no_warn_on_missing_unsafe` were replaced by `--cfg linktime_*` flags — neither was used here. #### Verification - `cargo check` / `cargo clippy` clean across `turbo-tasks`, `turbo-tasks-backend`, `turbo-tasks-macros`, `turbo-tasks-macros-tests`, and `turbopack-tests`. - `cargo test -p turbo-tasks-backend`: all tests pass (58 in the largest suite, 0 failures across all integration tests, 0 doc-test failures). - `cargo test -p turbopack-tests`: 218 + 87 tests pass, 0 failures, 1 ignored. #### Lockfile changes `Cargo.lock` reflects the new dep graph: `ctor` jumps to `1.0.6`, `link-section` to `0.17.2`, a new `linktime-proc-macro` is pulled in, and the old `ctor-proc-macro` / `dtor` / `dtor-proc-macro` entries are dropped (they were transitive deps of `ctor` 0.10). Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude Co-authored-by: Tobias Koppers --- Cargo.lock | 42 ++++++------------- Cargo.toml | 2 +- .../tests/trybuild.rs | 2 +- .../src/value_impl_macro.rs | 30 ++++++------- 4 files changed, 30 insertions(+), 46 deletions(-) diff --git a/Cargo.lock b/Cargo.lock index ed70eb1d8245..21e66a10ba10 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -2007,21 +2007,14 @@ dependencies = [ [[package]] name = "ctor" -version = "0.10.1" +version = "1.0.6" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "83cf0d42651b16c6dfe68685716d18480d18a9c39c62d76e8cf3eb6ed5d8bcbf" +checksum = "6d765eb1c0bda10d31e0ea185f5ee15da532d60b0912d2bd1441783439e749c5" dependencies = [ - "ctor-proc-macro", - "dtor", "link-section", + "linktime-proc-macro", ] -[[package]] -name = "ctor-proc-macro" -version = "0.0.13" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "7a949c44fcacbbbb7ada007dc7acb34603dd97cd47de5d054f2b6493ecebb483" - [[package]] name = "cty" version = "0.2.2" @@ -2387,21 +2380,6 @@ dependencies = [ "dtoa", ] -[[package]] -name = "dtor" -version = "0.8.1" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "edf234dd1594d6dd434a8fb8cada51ddbbc593e40e4a01556a0b31c62da2775b" -dependencies = [ - "dtor-proc-macro", -] - -[[package]] -name = "dtor-proc-macro" -version = "0.0.13" -source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2647271c92754afcb174e758003cfd1cbf1e43e5a7853d7b1813e63e19e39a73" - [[package]] name = "dunce" version = "1.0.4" @@ -4200,9 +4178,9 @@ dependencies = [ [[package]] name = "link-section" -version = "0.2.1" +version = "0.17.2" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "b685d66585d646efe09fec763d796c291049c8b6bf84e04954bffc8748341f0d" +checksum = "4d1e908a416d6e9f725743b84a36feea40c4c131e805fbc26d61f9f451f36080" [[package]] name = "linked-hash-map" @@ -4219,6 +4197,12 @@ dependencies = [ "linked-hash-map", ] +[[package]] +name = "linktime-proc-macro" +version = "0.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a44cd706ff0d503ee32b2071166510ca27e281228de10cd3aa8d35ff94560f81" + [[package]] name = "linux-raw-sys" version = "0.4.14" @@ -9825,7 +9809,7 @@ dependencies = [ "bincode 2.0.1", "codspeed-criterion-compat", "concurrent-queue", - "ctor 0.10.1", + "ctor 1.0.6", "dashmap 6.1.0", "either", "erased-serde", @@ -10050,7 +10034,7 @@ version = "0.1.0" dependencies = [ "anyhow", "bincode 2.0.1", - "ctor 0.10.1", + "ctor 1.0.6", "serde", "tokio", "trybuild", diff --git a/Cargo.toml b/Cargo.toml index a1d6ec2fb973..8ae5fc594d10 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -258,7 +258,7 @@ console-subscriber = "0.4.1" const_format = "0.2.30" crc32fast = "1.5.0" criterion = { package = "codspeed-criterion-compat", version = "4.3.0" } -ctor = "0.10" +ctor = "1.0.6" crossbeam-channel = "0.5.8" crossbeam-utils = "0.8" dashmap = "6.1.0" diff --git a/turbopack/crates/turbo-tasks-macros-tests/tests/trybuild.rs b/turbopack/crates/turbo-tasks-macros-tests/tests/trybuild.rs index 80b50bdf4fc6..d1dbc7db51f8 100644 --- a/turbopack/crates/turbo-tasks-macros-tests/tests/trybuild.rs +++ b/turbopack/crates/turbo-tasks-macros-tests/tests/trybuild.rs @@ -3,7 +3,7 @@ // trybuild's stderr snapshot comparisons. Unsetting it here means only the // sub-compilations trybuild spawns are affected — the main test binary was already // compiled with sccache. -#[ctor::ctor] +#[ctor::ctor(unsafe)] fn unset_rustc_wrapper() { unsafe { std::env::remove_var("RUSTC_WRAPPER") }; } diff --git a/turbopack/crates/turbo-tasks-macros/src/value_impl_macro.rs b/turbopack/crates/turbo-tasks-macros/src/value_impl_macro.rs index 5a739cbf184d..7d2b19d30da7 100644 --- a/turbopack/crates/turbo-tasks-macros/src/value_impl_macro.rs +++ b/turbopack/crates/turbo-tasks-macros/src/value_impl_macro.rs @@ -317,21 +317,21 @@ pub fn value_impl(args: TokenStream, input: TokenStream) -> TokenStream { // there's no runtime `transmute` or indirect fn call. #[cfg(not(rust_analyzer))] - #[turbo_tasks::macro_helpers::ctor::ctor( - crate_path = turbo_tasks::macro_helpers::ctor, - )] - #[allow(non_snake_case)] - fn #vtable_register_ident() { - <::std::boxed::Box as turbo_tasks::VcValueTrait>::IMPL_VTABLES - .register( - <#ty as turbo_tasks::macro_helpers::RegistryDef::>::DEF, - { - let p: *const #ty = ::std::ptr::null(); - // This attaches a fat pointer to the null pointer. - let fat: *const dyn #trait_path = p; - fat - }, - ); + turbo_tasks::macro_helpers::ctor::declarative::ctor! { + #[ctor(unsafe)] + #[allow(non_snake_case)] + fn #vtable_register_ident() { + <::std::boxed::Box as turbo_tasks::VcValueTrait>::IMPL_VTABLES + .register( + <#ty as turbo_tasks::macro_helpers::RegistryDef::>::DEF, + { + let p: *const #ty = ::std::ptr::null(); + // This attaches a fat pointer to the null pointer. + let fat: *const dyn #trait_path = p; + fat + }, + ); + } } // NOTE(alexkirsz) We can't have a general `turbo_tasks::Upcast> for T where T: Trait` because From 5200800187569060b399d73b486a1bde9ef1f3cb Mon Sep 17 00:00:00 2001 From: Tobias Koppers Date: Mon, 1 Jun 2026 14:26:05 +0200 Subject: [PATCH 3/4] Include `--port` in `next internal query-trace` startup hint and help example (#93961) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ### What? When the Turbopack trace server starts (via `next internal trace `), it prints a hint suggesting users run `next internal query-trace --help` from another terminal. That suggestion doesn't include the actual MCP port the server is listening on, so anyone who copies it verbatim ends up querying the default port (`5748`) — which is wrong as soon as the server picked a non-default `--mcp-port`. This PR threads the running MCP port into both the startup hint and the `query-trace --help` output so the suggested command lines target the server that's actually running. Before: ``` Trace file opened Turbopack trace server started. View trace at https://trace.nextjs.org?port=5750 Query this trace from the command line: next internal query-trace --help Alternatively, connect an MCP client to http://127.0.0.1:5751/mcp ``` After: ``` Trace file opened Turbopack trace server started. View trace at https://trace.nextjs.org?port=5750 Query this trace from the command line: next internal query-trace --help --port 5751 Alternatively, connect an MCP client to http://127.0.0.1:5751/mcp ``` And `next internal query-trace --help --port 5751` now ends with: ``` Example: next internal query-trace --port 5751 --parent ``` ### Why? The MCP HTTP port is dynamic (defaults to `wsPort + 1`, and `--mcp-port` can override it). Without echoing it into the hint, the suggested follow-up command silently connects to the wrong port and the user gets a confusing "Could not connect to trace server on port 5748" error instead of a useful response. Surfacing the right `--port` value at the point the user copies the command keeps the discovery flow self-consistent. ### How? - `packages/next/src/cli/internal/turbo-trace-server.ts`: the startup hint now interpolates the actual `httpPort` the MCP server bound to, so it prints `next internal query-trace --help --port `. - `packages/next/src/bin/next.ts`: the `query-trace` command gains an `.addHelpText('after', …)` block that reads the parsed `--port` option (falling back to the default `5748`) and appends an `Example:` section using that value. Passing `--port ` alongside `--help` therefore produces an immediately-runnable example for that specific server. The `--port` CLI option itself, its default, and parsing/validation are unchanged. ### Verification Built `next` and ran: - `next internal query-trace --help` → example shows `--port 5748` (default). - `next internal query-trace --help --port 5751` → example shows `--port 5751`. - `next internal query-trace --port 5751 --help` → example shows `--port 5751`. The existing `test/e2e/turbopack-trace-server-query/turbopack-trace-server.test.ts` suite still matches the substrings it asserts on (`next internal trace ` and `next internal query-trace --help`). Co-authored-by: v-work-app[bot] <262237222+v-work-app[bot]@users.noreply.github.com> Co-authored-by: Claude Co-authored-by: Tobias Koppers --- packages/next/src/bin/next.ts | 4 ++++ packages/next/src/cli/internal/turbo-trace-server.ts | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/packages/next/src/bin/next.ts b/packages/next/src/bin/next.ts index 0e7d68d76a25..a6abadfe89a8 100755 --- a/packages/next/src/bin/next.ts +++ b/packages/next/src/bin/next.ts @@ -669,6 +669,10 @@ internal parseValidPositiveInteger ) ) + .addHelpText('after', ({ command }) => { + const port = (command.opts() as { port?: number }).port ?? 5748 + return `\nExample:\n next internal query-trace --port ${port} --parent ` + }) .action((options) => import('../cli/internal/query-trace.js').then((mod) => mod.queryTraceCli(options) diff --git a/packages/next/src/cli/internal/turbo-trace-server.ts b/packages/next/src/cli/internal/turbo-trace-server.ts index 54d8aeb70b6f..1bbc550bf1ee 100644 --- a/packages/next/src/cli/internal/turbo-trace-server.ts +++ b/packages/next/src/cli/internal/turbo-trace-server.ts @@ -288,7 +288,7 @@ export async function startTurboTraceServerCli( server.listen(httpPort, '127.0.0.1', () => { console.log( - `Query this trace from the command line: next internal query-trace --help` + `Query this trace from the command line: next internal query-trace --help --port ${httpPort}` ) console.log( `Alternatively, connect an MCP client to http://127.0.0.1:${httpPort}/mcp` From 87ee4fe431114b7432544711302aed3fbacbd0c1 Mon Sep 17 00:00:00 2001 From: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com> Date: Mon, 1 Jun 2026 14:57:41 +0200 Subject: [PATCH 4/4] Turbopack: refactor NFT to add Endpoint.traced_files (#94224) Refactor the module-graph bit of NFT so that we can provide `Endpoint.traced_files()` which is needed for the bundle analyzer There should be no logic change whatsoever --- Cargo.lock | 2 + crates/next-api/src/app.rs | 36 +- crates/next-api/src/empty.rs | 6 + crates/next-api/src/instrumentation.rs | 33 +- crates/next-api/src/lib.rs | 1 + crates/next-api/src/middleware.rs | 33 +- crates/next-api/src/next_server_nft.rs | 41 +- crates/next-api/src/nft.rs | 394 +++++++++++++++ crates/next-api/src/nft_json.rs | 634 +++---------------------- crates/next-api/src/pages.rs | 26 +- crates/next-api/src/route.rs | 27 ++ crates/next-core/Cargo.toml | 4 + crates/next-core/src/next_config.rs | 76 ++- crates/next-core/src/util.rs | 207 +++++++- 14 files changed, 893 insertions(+), 627 deletions(-) create mode 100644 crates/next-api/src/nft.rs diff --git a/Cargo.lock b/Cargo.lock index 21e66a10ba10..4116cc31ed1e 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4826,11 +4826,13 @@ dependencies = [ "swc_core", "swc_sourcemap", "thiserror 1.0.69", + "tokio", "tracing", "turbo-bincode", "turbo-esregex", "turbo-rcstr", "turbo-tasks", + "turbo-tasks-backend", "turbo-tasks-bytes", "turbo-tasks-env", "turbo-tasks-fetch", diff --git a/crates/next-api/src/app.rs b/crates/next-api/src/app.rs index 7e5d0686cf35..178484c899a0 100644 --- a/crates/next-api/src/app.rs +++ b/crates/next-api/src/app.rs @@ -78,6 +78,7 @@ use crate::{ font::FontManifest, loadable_manifest::create_react_loadable_manifest, module_graph::{ClientReferencesGraphs, NextDynamicGraphs, ServerActionsGraphs}, + nft::{EndpointTraceResult, trace_endpoint}, nft_json::NftJsonAsset, paths::{ all_asset_paths, all_paths_in_root, get_asset_paths_from_root, get_js_paths_from_root, @@ -1789,8 +1790,7 @@ impl AppEndpoint { .chain(loadable_manifest_output.iter().flat_map(|m| &**m).copied()) .map(|m| *m) .collect(), - *module_graphs.full, - vec![*rsc_entry], + self.trace_result(), ) .to_resolved() .await?, @@ -1978,6 +1978,33 @@ impl AppEndpoint { } }) } + + #[turbo_tasks::function] + async fn trace_result(self: Vc) -> Result> { + let this = self.await?; + let app_entry = self.app_endpoint_entry().await?; + + let rsc_entry = app_entry.rsc_entry; + + let is_app_page = matches!(this.ty, AppEndpointType::Page { .. }); + + let module_graphs = this + .app_project + .app_module_graphs( + self, + *rsc_entry, + // We only need the client runtime entries for pages not for Route Handlers + is_app_page.then(|| this.app_project.client_runtime_entries()), + ) + .await?; + + Ok(trace_endpoint( + this.app_project.project(), + Some(app_function_name(&app_entry.original_name).into()), + *module_graphs.full, + vec![*rsc_entry], + )) + } } async fn create_app_paths_manifest( @@ -2200,6 +2227,11 @@ impl Endpoint for AppEndpoint { async fn project(self: Vc) -> Result> { Ok(self.await?.app_project.project()) } + + #[turbo_tasks::function] + fn traced_files(self: Vc) -> Vc { + self.trace_result().all_files() + } } #[turbo_tasks::value] diff --git a/crates/next-api/src/empty.rs b/crates/next-api/src/empty.rs index 6eb76d41f7d8..3b63d7f7595c 100644 --- a/crates/next-api/src/empty.rs +++ b/crates/next-api/src/empty.rs @@ -1,4 +1,5 @@ use anyhow::{Result, bail}; +use next_core::app_structure::FileSystemPathVec; use turbo_tasks::{Completion, ResolvedVc, Vc}; use turbopack_core::module_graph::GraphEntries; @@ -51,4 +52,9 @@ impl Endpoint for EmptyEndpoint { fn project(&self) -> Vc { *self.project } + + #[turbo_tasks::function] + fn traced_files(self: Vc) -> Vc { + Vc::cell(vec![]) + } } diff --git a/crates/next-api/src/instrumentation.rs b/crates/next-api/src/instrumentation.rs index a52e27a60b82..f8fb45e01e46 100644 --- a/crates/next-api/src/instrumentation.rs +++ b/crates/next-api/src/instrumentation.rs @@ -1,5 +1,6 @@ use anyhow::Result; use next_core::{ + app_structure::FileSystemPathVec, next_edge::entry::wrap_edge_entry, next_manifests::{InstrumentationDefinition, MiddlewaresManifestV2}, }; @@ -26,6 +27,7 @@ use turbopack_core::{ }; use crate::{ + nft::{EndpointTraceResult, trace_endpoint}, nft_json::NftJsonAsset, paths::{ all_asset_paths, get_js_paths_from_root, get_wasm_paths_from_root, wasm_paths_to_bindings, @@ -185,23 +187,27 @@ impl InstrumentationEndpoint { let chunk = self.node_chunk().to_resolved().await?; let mut output_assets = vec![chunk]; if this.project.next_mode().await?.is_production() { - let userland_module = self.entry_module(); output_assets.push(ResolvedVc::upcast( - NftJsonAsset::new( - *this.project, - None, - *chunk, - vec![], - this.project.module_graph(userland_module), - vec![userland_module], - ) - .to_resolved() - .await?, + NftJsonAsset::new(*this.project, None, *chunk, vec![], self.trace_result()) + .to_resolved() + .await?, )); } Ok(Vc::cell(output_assets)) } } + + #[turbo_tasks::function] + async fn trace_result(self: Vc) -> Result> { + let this = self.await?; + let userland_module = self.entry_module(); + Ok(trace_endpoint( + *this.project, + None, + this.project.module_graph(userland_module), + vec![userland_module], + )) + } } #[turbo_tasks::value_impl] @@ -265,4 +271,9 @@ impl Endpoint for InstrumentationEndpoint { fn project(&self) -> Vc { *self.project } + + #[turbo_tasks::function] + fn traced_files(self: Vc) -> Vc { + self.trace_result().all_files() + } } diff --git a/crates/next-api/src/lib.rs b/crates/next-api/src/lib.rs index cfa00193c203..c639e6ba5549 100644 --- a/crates/next-api/src/lib.rs +++ b/crates/next-api/src/lib.rs @@ -15,6 +15,7 @@ mod loadable_manifest; mod middleware; mod module_graph; pub mod next_server_nft; +mod nft; mod nft_json; pub mod operation; mod pages; diff --git a/crates/next-api/src/middleware.rs b/crates/next-api/src/middleware.rs index bea471ee4f06..b4472da46f2c 100644 --- a/crates/next-api/src/middleware.rs +++ b/crates/next-api/src/middleware.rs @@ -2,6 +2,7 @@ use std::future::IntoFuture; use anyhow::{Context, Result}; use next_core::{ + app_structure::FileSystemPathVec, middleware::get_middleware_module, next_edge::entry::wrap_edge_entry, next_manifests::{EdgeFunctionDefinition, MiddlewaresManifestV2, ProxyMatcher, Regions}, @@ -28,6 +29,7 @@ use turbopack_core::{ }; use crate::{ + nft::{EndpointTraceResult, trace_endpoint}, nft_json::NftJsonAsset, paths::{ all_asset_paths, all_paths_in_root, get_asset_paths_from_root, get_js_paths_from_root, @@ -227,18 +229,10 @@ impl MiddlewareEndpoint { let chunk = self.node_chunk().to_resolved().await?; let mut output_assets = vec![chunk]; if this.project.next_mode().await?.is_production() { - let userland_module = self.entry_module(); output_assets.push(ResolvedVc::upcast( - NftJsonAsset::new( - *this.project, - None, - *chunk, - vec![], - this.project.module_graph(userland_module), - vec![userland_module], - ) - .to_resolved() - .await?, + NftJsonAsset::new(*this.project, None, *chunk, vec![], self.trace_result()) + .to_resolved() + .await?, )); } let middleware_manifest_v2 = MiddlewaresManifestV2 { @@ -344,6 +338,18 @@ impl MiddlewareEndpoint { ) .module() } + + #[turbo_tasks::function] + async fn trace_result(self: Vc) -> Result> { + let this = self.await?; + let userland_module = self.entry_module(); + Ok(trace_endpoint( + *this.project, + None, + this.project.module_graph(userland_module), + vec![userland_module], + )) + } } #[turbo_tasks::value_impl] @@ -420,4 +426,9 @@ impl Endpoint for MiddlewareEndpoint { fn project(&self) -> Vc { *self.project } + + #[turbo_tasks::function] + fn traced_files(self: Vc) -> Vc { + self.trace_result().all_files() + } } diff --git a/crates/next-api/src/next_server_nft.rs b/crates/next-api/src/next_server_nft.rs index 83e9f70e4442..017abcd5346e 100644 --- a/crates/next-api/src/next_server_nft.rs +++ b/crates/next-api/src/next_server_nft.rs @@ -4,8 +4,7 @@ use anyhow::{Context, Result, bail}; use bincode::{Decode, Encode}; use either::Either; use next_core::{get_next_package, next_server::get_tracing_compile_time_info}; -use serde_json::{Value, json}; -use turbo_rcstr::RcStr; +use serde_json::json; use turbo_tasks::{ NonLocalValue, ResolvedVc, TaskInput, TryFlatJoinIterExt, TryJoinIterExt, Vc, trace::TraceRawVcs, @@ -27,10 +26,7 @@ use turbopack_core::{ }; use turbopack_resolve::ecmascript::cjs_resolve; -use crate::{ - nft_json::{relativize_glob, traced_modules_for_entries}, - project::Project, -}; +use crate::{nft::traced_modules_for_entries, project::Project}; #[derive( PartialEq, Eq, TraceRawVcs, NonLocalValue, Debug, Clone, Hash, TaskInput, Encode, Decode, @@ -304,30 +300,19 @@ impl ServerNftJsonAsset { let output_file_tracing_excludes = self .project .next_config() - .output_file_tracing_excludes() + .output_file_tracing_excludes(project_path) .await?; let mut additional_ignores = BTreeSet::new(); - if let Some(output_file_tracing_excludes) = output_file_tracing_excludes - .as_ref() - .and_then(Value::as_object) - { - for (glob_pattern, exclude_patterns) in output_file_tracing_excludes { - // Check if the route matches the glob pattern - let glob = Glob::new(RcStr::from(glob_pattern.clone()), Default::default()).await?; - if glob.matches("next-server") - && let Some(patterns) = exclude_patterns.as_array() - { - for pattern in patterns { - if let Some(pattern_str) = pattern.as_str() { - let (glob, root) = relativize_glob(pattern_str, project_path.clone())?; - let glob = if root.path.is_empty() { - glob.to_string() - } else { - format!("{root}/{glob}") - }; - additional_ignores.insert(glob); - } - } + + for (route_glob, exclude_patterns) in output_file_tracing_excludes.iter() { + // Check if the route matches the glob pattern + if route_glob.await?.matches("next-server") { + for (glob, root) in exclude_patterns { + additional_ignores.insert(if root.path.is_empty() { + glob.to_string() + } else { + format!("{root}/{glob}") + }); } } } diff --git a/crates/next-api/src/nft.rs b/crates/next-api/src/nft.rs new file mode 100644 index 000000000000..5bb3cff66bc1 --- /dev/null +++ b/crates/next-api/src/nft.rs @@ -0,0 +1,394 @@ +use std::collections::{BTreeSet, VecDeque}; + +use anyhow::{Context, Result}; +use next_core::{app_structure::FileSystemPathVec, next_config::NextConfig}; +use rustc_hash::{FxHashMap, FxHashSet}; +use tracing::Instrument; +use turbo_rcstr::RcStr; +use turbo_tasks::{ + FxIndexMap, FxIndexSet, ReadRef, ResolvedVc, TryFlatJoinIterExt, TryJoinIterExt, Vc, +}; +use turbo_tasks_fs::{ + DirectoryEntry, FileSystemPath, + glob::{Glob, GlobOptions}, +}; +use turbo_tasks_hash::HashAlgorithm; +use turbopack_core::{ + asset::Asset, + chunk::{ChunkingType, TracedMode}, + ident::AssetIdent, + module::{Module, Modules}, + module_graph::{GraphTraversalAction, ModuleGraph}, +}; + +use crate::project::Project; + +#[turbo_tasks::value] +pub struct EndpointTraceResult { + pub modules: Vec>>, + pub includes: Vec, + pub module_data: ResolvedVc, +} + +#[turbo_tasks::value_impl] +impl EndpointTraceResult { + #[turbo_tasks::function] + pub async fn all_files(&self) -> Result> { + let module_data = self.module_data.await?; + Ok(Vc::cell( + self.includes + .iter() + .cloned() + .chain( + self.modules + .iter() + .map(async |m| Ok(module_data.idents.get(m).await?.unwrap().path.clone())) + .try_join() + .await?, + ) + .collect(), + )) + } +} + +#[turbo_tasks::function] +pub async fn trace_endpoint( + project: ResolvedVc, + page_name: Option, + module_graph: ResolvedVc, + entry_modules: Vec>>, +) -> Result> { + let span = tracing::info_span!("trace endpoint", path = debug(&page_name)); + async { + let project_path = project.project_path().owned().await?; + let next_config = project.next_config(); + + let output_file_tracing_includes = next_config + .output_file_tracing_includes(project_path.clone()) + .await?; + + // Collect referenced assets and externals from module graph + let all_modules = traced_modules_for_entries( + *module_graph, + Vc::cell(entry_modules.clone()), + tracing_exclude_glob(page_name.clone(), project_path.clone(), next_config) + .await? + .map(|v| *v), + false, + ) + .await?; + + let module_data = traced_module_data_for_graph(*module_graph, false) + .to_resolved() + .await?; + let module_paths = module_data.await?.idents; + + let modules = all_modules + .iter() + .copied() + .map(async |module| { + let entry = module_paths + .get(&module) + .await? + .context("missing path for module")?; + let referenced_chunk_path = &entry.path; + + if referenced_chunk_path.has_extension(".map") { + return Ok(None); + } + + #[cfg(debug_assertions)] + { + // Verify that we there are no entries where a file is created inside of a + // symlink, as this can result in invalid ZIP files and deployment failures. For + // example + // node_modules/.pnpm/node_modules/@libsql/client/src/index.json + // where + // node_modules/.pnpm/node_modules/@libsql/client is a symlink + let parent_path = referenced_chunk_path.parent(); + if parent_path.realpath().await? != parent_path { + turbo_tasks::turbobail!( + "Encountered file inside of symlink in NFT list: {parent_path} is a \ + symlink, but {referenced_chunk_path} was created inside of it" + ); + } + } + + Ok(Some(module)) + }) + .try_flat_join() + .await?; + + // Apply outputFileTracingIncludes + // Extract route from chunk path for pattern matching + let includes = if let Some(route) = &page_name { + let mut combined_includes_by_root: FxIndexMap> = + FxIndexMap::default(); + + for (route_glob, include_patterns) in output_file_tracing_includes.iter() { + if route_glob.await?.matches(route) { + for (glob, root) in include_patterns { + combined_includes_by_root + .entry(root.clone()) + .or_default() + .push(glob); + } + } + } + + // Apply includes - find additional files that match the include patterns + let includes = combined_includes_by_root + .into_iter() + .map(|(root, globs)| { + let glob = Glob::new( + format!("{{{}}}", globs.join(",")).into(), + GlobOptions { contains: true }, + ); + get_glob_includes(root, glob) + }) + .try_join() + .await?; + + includes.into_iter().flatten().collect() + } else { + Default::default() + }; + + Ok(EndpointTraceResult { + modules, + includes, + module_data, + } + .cell()) + } + .instrument(span) + .await +} + +/// Apply outputFileTracingIncludes patterns to find additional files +async fn get_glob_includes( + project_root_path: FileSystemPath, + glob: Vc, +) -> Result> { + // Read files matching the glob pattern from the project root + // DETERMINISM: the sort_by call below ensures determinism. + let glob_result = project_root_path.read_glob(glob).await?; + + // Walk the full glob_result using an explicit stack to avoid async recursion overheads. + // Use a BTreeSet to get determinstic order (return value of `read_glob` has random order). + let mut result = vec![]; + let mut stack = VecDeque::new(); + stack.push_back(glob_result); + while let Some(glob_result) = stack.pop_back() { + // Process direct results (files and directories at this level) + for entry in glob_result.results.values() { + let (DirectoryEntry::File(file_path) | DirectoryEntry::Symlink(file_path)) = entry + else { + continue; + }; + + result.push(file_path.clone()); + } + + for nested_result in glob_result.inner.values() { + let nested_result_ref = nested_result.await?; + stack.push_back(nested_result_ref); + } + } + + // All paths were matched from project_root_path, so they must all have the same `fs`. So it's + // enough to sort by path. + result.sort_by(|a, b| a.path.cmp(&b.path)); + + Ok(result) +} + +#[turbo_tasks::value(transparent)] +pub struct OptionGlob(Option>); + +#[turbo_tasks::function] +pub async fn tracing_exclude_glob( + page_name: Option, + project_path: FileSystemPath, + next_config: ResolvedVc, +) -> Result> { + Ok(if let Some(page_name) = &page_name { + let route = format!("/{page_name}"); + let output_file_tracing_excludes = next_config + .output_file_tracing_excludes(project_path) + .await?; + let mut combined_excludes = BTreeSet::new(); + + for (route_glob, exclude_patterns) in output_file_tracing_excludes.iter() { + if route_glob.await?.matches(&route) { + for (glob, root) in exclude_patterns { + combined_excludes.insert(if root.path.is_empty() { + glob.to_string() + } else { + format!("{root}/{glob}") + }); + } + } + } + + if combined_excludes.is_empty() { + Vc::cell(None) + } else { + let glob = Glob::new( + format!( + "{{{}}}", + combined_excludes + .iter() + .map(|s| s.as_str()) + .collect::>() + .join(",") + ) + .into(), + GlobOptions { contains: true }, + ) + .to_resolved() + .await?; + + Vc::cell(Some(glob)) + } + } else { + Vc::cell(None) + }) +} + +#[turbo_tasks::function] +pub async fn traced_modules_for_entries( + module_graph: Vc, + entry_modules: Vc, + exclude_glob: Option>, + entries_are_traced: bool, +) -> Result> { + let exclude_glob_and_module_idents = if let Some(exclude_glob) = exclude_glob { + let exclude_glob = exclude_glob.await?; + let data = traced_module_data_for_graph(module_graph, entries_are_traced).await?; + Some((exclude_glob, data.idents.await?)) + } else { + None + }; + + let mut traced_modules = FxIndexSet::default(); + module_graph.await?.traverse_edges_dfs( + entry_modules.await?.iter().copied(), + &mut (), + |parent, target, _| { + let Some((parent, ref_data)) = parent else { + if entries_are_traced { + traced_modules.insert(target); + } + return Ok(GraphTraversalAction::Continue); + }; + + if should_visit_for_tracing(&ref_data.chunking_type, traced_modules.contains(&parent)) { + if let Some((exclude_glob, module_idents)) = &exclude_glob_and_module_idents + && exclude_glob.matches( + &module_idents + .get(&target) + .context("missing path for module")? + .path + .path, + ) + { + return Ok(GraphTraversalAction::Skip); + } + traced_modules.insert(target); + }; + Ok(GraphTraversalAction::Continue) + }, + |_, _, _| Ok(()), + true, + )?; + + Ok(Vc::cell(traced_modules.into_iter().collect())) +} + +/// Ignore non-entry traced reference if not already in tracing mode. +/// +/// ChunkingType::Traced{TracedMode::Entry} => target is always traced +/// ChunkingType::Traced{TracedMode::Transitive} => target only traced if parent is traced +/// ChunkingType::* => target only traced if parent is traced +fn should_visit_for_tracing(chunking_type: &ChunkingType, parent_traced: bool) -> bool { + matches!( + chunking_type, + ChunkingType::Traced { + mode: TracedMode::Entry + } + ) || parent_traced +} + +#[turbo_tasks::value(transparent, cell = "keyed")] +pub struct TracedModuleDataIdents(FxHashMap>, ReadRef>); + +#[turbo_tasks::value(transparent, cell = "keyed")] +pub struct TracedModuleDataHashes(FxHashMap>, ReadRef>); + +#[turbo_tasks::value] +pub struct TracedModuleData { + pub idents: ResolvedVc, + pub hashes: ResolvedVc, +} + +/// This caches the paths for all modules in the graph so that we don't have to do it once per page. +#[turbo_tasks::function] +pub async fn traced_module_data_for_graph( + module_graph: Vc, + entries_are_traced: bool, +) -> Result> { + // This function is very similar to traced_modules_for_entries, but doesn't apply the glob and + // is executed only once for the whole graph. + let module_graph = module_graph.await?; + let entries = module_graph.graphs.iter().flat_map(|g| g.entry_modules()); + + let mut traced_modules = FxHashSet::default(); + module_graph.traverse_edges_dfs( + entries, + &mut (), + |parent, target, _| { + let Some((parent, ref_data)) = parent else { + if entries_are_traced { + traced_modules.insert(target); + } + return Ok(GraphTraversalAction::Continue); + }; + + if should_visit_for_tracing(&ref_data.chunking_type, traced_modules.contains(&parent)) { + traced_modules.insert(target); + }; + Ok(GraphTraversalAction::Continue) + }, + |_, _, _| Ok(()), + true, + )?; + + let (idents, hashes): (FxHashMap<_, _>, FxHashMap<_, _>) = traced_modules + .into_iter() + .map(async |module| { + Ok(( + (module, module.ident().await?), + ( + module, + module + .source() + .await? + .context("NFT module has no content")? + .content() + .hash(HashAlgorithm::Xxh3Hash128Hex) + .await?, + ), + )) + }) + .try_join() + .await? + .into_iter() + .unzip(); + + Ok(TracedModuleData { + idents: ResolvedVc::cell(idents), + hashes: ResolvedVc::cell(hashes), + } + .cell()) +} diff --git a/crates/next-api/src/nft_json.rs b/crates/next-api/src/nft_json.rs index 5878d338e5d5..70fdd5185786 100644 --- a/crates/next-api/src/nft_json.rs +++ b/crates/next-api/src/nft_json.rs @@ -1,36 +1,27 @@ -use std::collections::{BTreeMap, BTreeSet, VecDeque}; - use anyhow::{Context, Result, bail}; use async_trait::async_trait; -use bincode::{Decode, Encode}; use either::Either; -use rustc_hash::{FxHashMap, FxHashSet}; use serde_json::json; use tracing::{Instrument, Level, Span}; use turbo_rcstr::{RcStr, rcstr}; use turbo_tasks::{ - FxIndexMap, FxIndexSet, NonLocalValue, ReadRef, ResolvedVc, TryFlatJoinIterExt, TryJoinIterExt, - ValueToString, Vc, + ReadRef, ResolvedVc, TryFlatJoinIterExt, TryJoinIterExt, ValueToString, Vc, graph::{AdjacencyMap, GraphTraversal, Visit}, - trace::TraceRawVcs, turbofmt, }; -use turbo_tasks_fs::{ - DirectoryEntry, File, FileContent, FileSystem, FileSystemPath, - glob::{Glob, GlobOptions}, -}; +use turbo_tasks_fs::{File, FileContent, FileSystem, FileSystemPath, glob::Glob}; use turbo_tasks_hash::HashAlgorithm; use turbopack_core::{ asset::{Asset, AssetContent}, - chunk::{ChunkingType, TracedMode}, - ident::AssetIdent, issue::{Issue, IssueExt, IssueSeverity, IssueStage, StyledString}, - module::{Module, Modules}, - module_graph::{GraphTraversalAction, ModuleGraph}, + module::Module, output::{OutputAsset, OutputAssets, OutputAssetsReference}, }; -use crate::project::Project; +use crate::{ + nft::{EndpointTraceResult, tracing_exclude_glob}, + project::Project, +}; /// A json file that produces references to all files that are needed by the given module /// at runtime. This will include, for example, node native modules, unanalyzable packages, @@ -50,10 +41,9 @@ pub struct NftJsonAsset { /// next.js. additional_assets: Vec>>, // The page name, e.g. `pages/index` or `app/route1` - page_name: Option, + page_name: Option, - module_graph: ResolvedVc, - entry_modules: Vec>>, + traced_files: ResolvedVc, } #[turbo_tasks::value_impl] @@ -64,16 +54,14 @@ impl NftJsonAsset { page_name: Option, chunk: ResolvedVc>, additional_assets: Vec>>, - module_graph: ResolvedVc, - entry_modules: Vec>>, + traced_files: ResolvedVc, ) -> Vc { NftJsonAsset { chunk, project, additional_assets, - page_name: page_name.map(|page_name| format!("/{page_name}")), - module_graph, - entry_modules, + page_name, + traced_files, } .cell() } @@ -118,47 +106,6 @@ fn get_output_specifier( bail!("NftJsonAsset: cannot handle filepath '{path_ref}'"); } -/// Apply outputFileTracingIncludes patterns to find additional files -async fn apply_includes( - project_root_path: FileSystemPath, - glob: Vc, - ident_folder: &FileSystemPath, -) -> Result>> { - debug_assert_eq!(project_root_path.fs, ident_folder.fs); - // Read files matching the glob pattern from the project root - // This result itself has random order, but the BTreeSet will ensure a deterministic ordering. - let glob_result = project_root_path.read_glob(glob).await?; - - // Walk the full glob_result using an explicit stack to avoid async recursion overheads. - let mut result = BTreeMap::new(); - let mut stack = VecDeque::new(); - stack.push_back(glob_result); - while let Some(glob_result) = stack.pop_back() { - // Process direct results (files and directories at this level) - for entry in glob_result.results.values() { - let (DirectoryEntry::File(file_path) | DirectoryEntry::Symlink(file_path)) = entry - else { - continue; - }; - - // Convert to relative path from ident_folder to the file - // unwrap is safe because project_root_path and ident_folder have the same filesystem - // and paths produced by read_glob stay in the filesystem - let relative_path = ident_folder.get_relative_path_to(file_path).unwrap(); - result.insert( - relative_path, - file_path.read().hash(HashAlgorithm::Xxh3Hash128Hex).await?, - ); - } - - for nested_result in glob_result.inner.values() { - let nested_result_ref = nested_result.await?; - stack.push_back(nested_result_ref); - } - } - Ok(result) -} - #[turbo_tasks::value_impl] impl Asset for NftJsonAsset { #[turbo_tasks::function] @@ -180,9 +127,6 @@ impl Asset for NftJsonAsset { .config_file_path(project_path.clone()) .await?; - let output_file_tracing_includes = &*next_config.output_file_tracing_includes().await?; - let output_file_tracing_excludes = &*next_config.output_file_tracing_excludes().await?; - let client_root = this.project.client_fs().root(); let client_root = client_root.owned().await?; @@ -201,61 +145,8 @@ impl Asset for NftJsonAsset { .chain(std::iter::once(chunk)) .collect(); - let exclude_glob = if let Some(route) = &this.page_name { - if let Some(excludes_config) = output_file_tracing_excludes { - let mut combined_excludes = BTreeSet::new(); - - if let Some(excludes_obj) = excludes_config.as_object() { - for (glob_pattern, exclude_patterns) in excludes_obj { - // Check if the route matches the glob pattern - let glob = Glob::new( - RcStr::from(glob_pattern.clone()), - GlobOptions { contains: true }, - ) - .await?; - if glob.matches(route) - && let Some(patterns) = exclude_patterns.as_array() - { - for pattern in patterns { - if let Some(pattern_str) = pattern.as_str() { - let (glob, root) = - relativize_glob(pattern_str, project_path.clone())?; - let glob = if root.path.is_empty() { - glob.to_string() - } else { - format!("{root}/{glob}") - }; - combined_excludes.insert(glob); - } - } - } - } - } - - if combined_excludes.is_empty() { - None - } else { - let glob = Glob::new( - format!( - "{{{}}}", - combined_excludes - .iter() - .map(|s| s.as_str()) - .collect::>() - .join(",") - ) - .into(), - GlobOptions { contains: true }, - ); - - Some(glob) - } - } else { - None - } - } else { - None - }; + let exclude_glob = + tracing_exclude_glob(this.page_name.clone(), project_path.clone(), next_config); enum AssetOrModule { Asset(ResolvedVc>), @@ -266,38 +157,43 @@ impl Asset for NftJsonAsset { let all_assets = all_assets_from_entries_filtered( Vc::cell(entries), Some(client_root.clone()), - exclude_glob, - ) - .await?; - // Collect referenced assets and externals from module graph - let all_modules = traced_modules_for_entries( - *this.module_graph, - Vc::cell(this.entry_modules.clone()), - exclude_glob, - false, + exclude_glob.await?.map(|v| *v), ) .await?; - let module_paths = traced_module_data_for_graph(*this.module_graph, false); + let traced_files = this.traced_files.await?; + let module_data = traced_files.module_data.await?; - let mut result: Vec<(RcStr, ReadRef)> = all_assets + let mut result: Vec<(RcStr, _)> = all_assets .iter() .filter(|a| **a != chunk) .copied() .map(AssetOrModule::Asset) - .chain(all_modules.iter().copied().map(AssetOrModule::Module)) + .chain( + traced_files + .modules + .iter() + .copied() + .map(AssetOrModule::Module), + ) .map(async |referenced| { let (referenced_chunk_path, hash) = match referenced { AssetOrModule::Asset(v) => ( Either::Left(v.path().await?), - v.content().hash(HashAlgorithm::Xxh3Hash128Hex).await?, + Either::Left(v.content().hash(HashAlgorithm::Xxh3Hash128Hex).await?), ), AssetOrModule::Module(v) => { - let entry = module_paths + let ident = module_data + .idents .get(&v) .await? .context("missing path for module")?; - (Either::Right(entry.ident.path.clone()), entry.hash.clone()) + let hash = module_data + .hashes + .get(&v) + .await? + .context("missing hash for module")?; + (Either::Right(ident.path.clone()), Either::Right(hash)) } }; let referenced_chunk_path = match &referenced_chunk_path { @@ -305,13 +201,13 @@ impl Asset for NftJsonAsset { Either::Right(p) => p, }; - if let AssetOrModule::Module(referenced) = referenced + if let AssetOrModule::Module(_) = referenced && referenced_chunk_path == &*next_config_path { // If next.config.js was traced, assume that the whole project was traced // (unintentionally). Print a message in this case to avoid deploying // unnecessary files. - ForbiddenTracedFileIssue::new(*referenced) + ForbiddenTracedFileIssue::new(referenced_chunk_path.clone()) .to_resolved() .await? .emit(); @@ -321,37 +217,6 @@ impl Asset for NftJsonAsset { return Ok(None); } - #[cfg(debug_assertions)] - { - // Verify that we there are no entries where a file is created inside of a - // symlink, as this can result in invalid ZIP files and - // deployment failures. For example - // node_modules/.pnpm/node_modules/@libsql/client/package.json - // where - // node_modules/.pnpm/node_modules/@libsql/client is a symlink - let mut current_path = referenced_chunk_path.parent(); - loop { - use turbo_tasks_fs::FileSystemEntryType; - - if current_path.is_root() { - break; - } - - if matches!( - &*current_path.get_type().await?, - FileSystemEntryType::Symlink - ) { - turbo_tasks::turbobail!( - "Encountered file inside of symlink in NFT list: \ - {current_path} is a symlink, but {referenced_chunk_path} was \ - created inside of it" - ); - } - - current_path = current_path.parent(); - } - } - let specifier = match get_output_specifier( referenced_chunk_path, &ident_folder, @@ -378,60 +243,42 @@ impl Asset for NftJsonAsset { .try_flat_join() .await?; - // Apply outputFileTracingIncludes and outputFileTracingExcludes - // Extract route from chunk path for pattern matching - if let Some(route) = &this.page_name { - let mut combined_includes_by_root: FxIndexMap> = - FxIndexMap::default(); - - // Process includes - if let Some(includes_config) = output_file_tracing_includes - && let Some(includes_obj) = includes_config.as_object() - { - for (glob_pattern, include_patterns) in includes_obj { - // Check if the route matches the glob pattern - let glob = - Glob::new(glob_pattern.as_str().into(), GlobOptions { contains: true }) - .await?; - if glob.matches(route) - && let Some(patterns) = include_patterns.as_array() - { - for pattern in patterns { - if let Some(pattern_str) = pattern.as_str() { - let (glob, root) = - relativize_glob(pattern_str, project_path.clone())?; - combined_includes_by_root - .entry(root) - .or_default() - .push(glob); - } - } - } - } - } - - // Apply includes - find additional files that match the include patterns - let includes = combined_includes_by_root - .into_iter() - .map(|(root, globs)| { - let glob = Glob::new( - format!("{{{}}}", globs.join(",")).into(), - GlobOptions { contains: true }, - ); - apply_includes(root, glob, &ident_folder_in_project_fs) + result.extend( + traced_files + .includes + .iter() + .map(async |file_path| { + let relative_path = ident_folder_in_project_fs + .get_relative_path_to(file_path) + .unwrap(); + Ok(( + relative_path, + Either::Left( + file_path.read().hash(HashAlgorithm::Xxh3Hash128Hex).await?, + ), + )) }) .try_join() - .await?; - - result.extend(includes.into_iter().flatten()); - } + .await?, + ); // Some of the output assets may have been included multiple times (in multiple chunking // contexts), or asset contexts. result.sort_unstable(); result.dedup(); - let (files, file_hashes): (Vec<_>, Vec<_>) = result.into_iter().unzip(); + let (files, file_hashes): (Vec<_>, Vec<_>) = result + .iter() + .map(|(name, hash)| { + ( + name, + match hash { + Either::Left(v) => &**v, + Either::Right(v) => &**v, + }, + ) + }) + .unzip(); // We can't just add this into "files" because Next.js sometimes decides to delete // output files such as `.next/server/pages/index.js` if that page was prerendered and // is fully static. An alternative would be to postprocess the nft file so that @@ -455,176 +302,10 @@ impl Asset for NftJsonAsset { } } -/// Ignore non-entry traced reference if not already in tracing mode. -/// -/// ChunkingType::Traced{TracedMode::Entry} => target is always traced -/// ChunkingType::Traced{TracedMode::Transitive} => target only traced if parent is traced -/// ChunkingType::* => target only traced if parent is traced -fn should_visit_for_tracing(chunking_type: &ChunkingType, parent_traced: bool) -> bool { - matches!( - chunking_type, - ChunkingType::Traced { - mode: TracedMode::Entry - } - ) || parent_traced -} - -#[turbo_tasks::function] -pub async fn traced_modules_for_entries( - module_graph: Vc, - entry_modules: Vc, - exclude_glob: Option>, - entries_are_traced: bool, -) -> Result> { - let exclude_glob = if let Some(exclude_glob) = exclude_glob { - Some(exclude_glob.await?) - } else { - None - }; - let module_paths = if exclude_glob.is_some() { - Some(traced_module_data_for_graph(module_graph, entries_are_traced).await?) - } else { - None - }; - - let mut traced_modules = FxIndexSet::default(); - module_graph.await?.traverse_edges_dfs( - entry_modules.await?.iter().copied(), - &mut (), - |parent, target, _| { - let Some((parent, ref_data)) = parent else { - if entries_are_traced { - traced_modules.insert(target); - } - return Ok(GraphTraversalAction::Continue); - }; - - if should_visit_for_tracing(&ref_data.chunking_type, traced_modules.contains(&parent)) { - if let Some(exclude_glob) = &exclude_glob - && exclude_glob.matches( - &module_paths - .as_ref() - .unwrap() - .get(&target) - .context("missing path for module")? - .ident - .path - .path, - ) - { - return Ok(GraphTraversalAction::Skip); - } - traced_modules.insert(target); - }; - Ok(GraphTraversalAction::Continue) - }, - |_, _, _| Ok(()), - true, - )?; - - Ok(Vc::cell(traced_modules.into_iter().collect())) -} - -#[derive(Clone, Debug, PartialEq, Eq, Hash, Encode, Decode, NonLocalValue, TraceRawVcs)] -struct TracedModuleData { - ident: ReadRef, - hash: ReadRef, -} - -#[turbo_tasks::value(transparent, cell = "keyed")] -struct TracedModuleDataMap(FxHashMap>, TracedModuleData>); - -/// This caches the paths for all modules in the graph so that we don't have to do it once per page. -#[turbo_tasks::function] -async fn traced_module_data_for_graph( - module_graph: Vc, - entries_are_traced: bool, -) -> Result> { - // This function is very similar to traced_modules_for_entries, but doesn't apply the glob and - // is executed only once for the whole graph. - let module_graph = module_graph.await?; - let entries = module_graph.graphs.iter().flat_map(|g| g.entry_modules()); - - let mut traced_modules = FxHashSet::default(); - module_graph.traverse_edges_dfs( - entries, - &mut (), - |parent, target, _| { - let Some((parent, ref_data)) = parent else { - if entries_are_traced { - traced_modules.insert(target); - } - return Ok(GraphTraversalAction::Continue); - }; - - if should_visit_for_tracing(&ref_data.chunking_type, traced_modules.contains(&parent)) { - traced_modules.insert(target); - }; - Ok(GraphTraversalAction::Continue) - }, - |_, _, _| Ok(()), - true, - )?; - - Ok(Vc::cell( - traced_modules - .into_iter() - .map(async |module| { - Ok(( - module, - TracedModuleData { - ident: module.ident().await?, - hash: module - .source() - .await? - .context("NFT module has no content")? - .content() - .hash(HashAlgorithm::Xxh3Hash128Hex) - .await?, - }, - )) - }) - .try_join() - .await? - .into_iter() - .collect(), - )) -} - -/// The globs defined in the next.config.mjs are relative to the project root. -/// The glob walker in turbopack is somewhat naive so we handle relative path directives first so -/// traversal doesn't need to consider them and can just traverse 'down' the tree. -/// The main alternative is to merge glob evaluation with directory traversal which is what the npm -/// `glob` package does, but this would be a substantial rewrite. -pub(crate) fn relativize_glob( - glob: &str, - relative_to: FileSystemPath, -) -> Result<(&str, FileSystemPath)> { - let mut relative_to = relative_to; - let mut processed_glob = glob; - loop { - if let Some(stripped) = processed_glob.strip_prefix("../") { - if relative_to.path.is_empty() { - bail!( - "glob '{glob}' is invalid, it has a prefix that navigates out of the project \ - root" - ); - } - relative_to = relative_to.parent(); - processed_glob = stripped; - } else if let Some(stripped) = processed_glob.strip_prefix("./") { - processed_glob = stripped; - } else { - break; - } - } - Ok((processed_glob, relative_to)) -} - /// Walks the asset graph from multiple assets and collect all referenced /// assets, but filters out all client assets and glob matches. #[turbo_tasks::function] -pub async fn all_assets_from_entries_filtered( +async fn all_assets_from_entries_filtered( entries: Vc, client_root: Option, exclude_glob: Option>, @@ -671,14 +352,14 @@ pub async fn all_assets_from_entries_filtered( #[turbo_tasks::value(shared)] struct ForbiddenTracedFileIssue { - module: ResolvedVc>, + file: FileSystemPath, } #[turbo_tasks::value_impl] impl ForbiddenTracedFileIssue { #[turbo_tasks::function] - pub fn new(module: ResolvedVc>) -> Vc { - Self { module }.cell() + pub fn new(file: FileSystemPath) -> Vc { + Self { file }.cell() } } @@ -696,7 +377,7 @@ impl Issue for ForbiddenTracedFileIssue { } async fn file_path(&self) -> Result { - Ok(self.module.ident().await?.path.clone()) + Ok(self.file.clone()) } async fn title(&self) -> Result { @@ -831,180 +512,3 @@ async fn get_referenced_server_assets( .try_flat_join() .await } - -#[cfg(test)] -mod tests { - use turbo_tasks::ResolvedVc; - use turbo_tasks_backend::{BackendOptions, TurboTasksBackend, noop_backing_storage}; - use turbo_tasks_fs::{FileSystemPath, NullFileSystem}; - - use super::*; - - fn create_test_fs_path(path: &str) -> FileSystemPath { - FileSystemPath { - fs: ResolvedVc::upcast(NullFileSystem {}.resolved_cell()), - path: path.into(), - } - } - - #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn test_relativize_glob_normal_patterns() { - let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( - BackendOptions::default(), - noop_backing_storage(), - )); - tt.run_once(async { - // Test normal glob patterns without relative prefixes - let base_path = create_test_fs_path("project/src"); - - let (glob, path) = relativize_glob("*.js", base_path.clone()).unwrap(); - assert_eq!(glob, "*.js"); - assert_eq!(path.path.as_str(), "project/src"); - - let (glob, path) = relativize_glob("components/**/*.tsx", base_path.clone()).unwrap(); - assert_eq!(glob, "components/**/*.tsx"); - assert_eq!(path.path.as_str(), "project/src"); - - let (glob, path) = relativize_glob("lib/utils.ts", base_path.clone()).unwrap(); - assert_eq!(glob, "lib/utils.ts"); - assert_eq!(path.path.as_str(), "project/src"); - Ok(()) - }) - .await - .unwrap(); - } - - #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn test_relativize_glob_current_directory_prefix() { - let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( - BackendOptions::default(), - noop_backing_storage(), - )); - tt.run_once(async { - let base_path = create_test_fs_path("project/src"); - - // Single ./ prefix - let (glob, path) = relativize_glob("./components/*.tsx", base_path.clone()).unwrap(); - assert_eq!(glob, "components/*.tsx"); - assert_eq!(path.path.as_str(), "project/src"); - - // Multiple ./ prefixes - let (glob, path) = relativize_glob("././utils.js", base_path.clone()).unwrap(); - assert_eq!(glob, "utils.js"); - assert_eq!(path.path.as_str(), "project/src"); - - // ./ with complex glob - let (glob, path) = relativize_glob("./lib/**/*.{js,ts}", base_path.clone()).unwrap(); - assert_eq!(glob, "lib/**/*.{js,ts}"); - assert_eq!(path.path.as_str(), "project/src"); - Ok(()) - }) - .await - .unwrap(); - } - - #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn test_relativize_glob_parent_directory_navigation() { - let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( - BackendOptions::default(), - noop_backing_storage(), - )); - tt.run_once(async { - let base_path = create_test_fs_path("project/src/components"); - - // Single ../ prefix - let (glob, path) = relativize_glob("../utils/*.js", base_path.clone()).unwrap(); - assert_eq!(glob, "utils/*.js"); - assert_eq!(path.path.as_str(), "project/src"); - - // Multiple ../ prefixes - let (glob, path) = relativize_glob("../../lib/*.ts", base_path.clone()).unwrap(); - assert_eq!(glob, "lib/*.ts"); - assert_eq!(path.path.as_str(), "project"); - - // Complex navigation with glob - let (glob, path) = - relativize_glob("../../../external/**/*.json", base_path.clone()).unwrap(); - assert_eq!(glob, "external/**/*.json"); - assert_eq!(path.path.as_str(), ""); - Ok(()) - }) - .await - .unwrap(); - } - - #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn test_relativize_glob_mixed_prefixes() { - let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( - BackendOptions::default(), - noop_backing_storage(), - )); - tt.run_once(async { - let base_path = create_test_fs_path("project/src/components"); - - // ../ followed by ./ - let (glob, path) = relativize_glob(".././utils/*.js", base_path.clone()).unwrap(); - assert_eq!(glob, "utils/*.js"); - assert_eq!(path.path.as_str(), "project/src"); - - // ./ followed by ../ - let (glob, path) = relativize_glob("./../lib/*.ts", base_path.clone()).unwrap(); - assert_eq!(glob, "lib/*.ts"); - assert_eq!(path.path.as_str(), "project/src"); - - // Multiple mixed prefixes - let (glob, path) = - relativize_glob("././../.././external/*.json", base_path.clone()).unwrap(); - assert_eq!(glob, "external/*.json"); - assert_eq!(path.path.as_str(), "project"); - Ok(()) - }) - .await - .unwrap(); - } - - #[tokio::test(flavor = "multi_thread", worker_threads = 2)] - async fn test_relativize_glob_error_navigation_out_of_root() { - let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( - BackendOptions::default(), - noop_backing_storage(), - )); - tt.run_once(async { - // Test navigating out of project root with empty path - let empty_path = create_test_fs_path(""); - let result = relativize_glob("../outside.js", empty_path); - assert!(result.is_err()); - assert!( - result - .unwrap_err() - .to_string() - .contains("navigates out of the project root") - ); - - // Test navigating too far up from a shallow path - let shallow_path = create_test_fs_path("project"); - let result = relativize_glob("../../outside.js", shallow_path); - assert!(result.is_err()); - assert!( - result - .unwrap_err() - .to_string() - .contains("navigates out of the project root") - ); - - // Test multiple ../ that would go out of root - let base_path = create_test_fs_path("a/b"); - let result = relativize_glob("../../../outside.js", base_path); - assert!(result.is_err()); - assert!( - result - .unwrap_err() - .to_string() - .contains("navigates out of the project root") - ); - Ok(()) - }) - .await - .unwrap(); - } -} diff --git a/crates/next-api/src/pages.rs b/crates/next-api/src/pages.rs index 4b319c6714c1..ac471602e6f7 100644 --- a/crates/next-api/src/pages.rs +++ b/crates/next-api/src/pages.rs @@ -2,7 +2,9 @@ use anyhow::{Context, Result, bail}; use bincode::{Decode, Encode}; use futures::future::BoxFuture; use next_core::{ - PageLoaderAsset, create_page_loader_entry_module, get_asset_path_from_pathname, + PageLoaderAsset, + app_structure::FileSystemPathVec, + create_page_loader_entry_module, get_asset_path_from_pathname, get_edge_resolve_options_context, hmr_entry::HmrEntryModule, mode::NextMode, @@ -73,6 +75,7 @@ use crate::{ font::FontManifest, loadable_manifest::create_react_loadable_manifest, module_graph::{NextDynamicGraphs, validate_pages_css_imports}, + nft::{EndpointTraceResult, trace_endpoint}, nft_json::NftJsonAsset, paths::{ all_asset_paths, all_paths_in_root, get_asset_paths_from_root, get_js_paths_from_root, @@ -1089,8 +1092,7 @@ impl PageEndpoint { Some(pages_function_name(&this.original_name).into()), *ssr_entry_chunk, additional_assets, - ssr_module_graph, - vec![*ssr_module], + self.trace_result(), ) .to_resolved() .await?, @@ -1557,6 +1559,19 @@ impl PageEndpoint { .await?, ))) } + + #[turbo_tasks::function] + async fn trace_result(self: Vc) -> Result> { + let this = self.await?; + let ssr_module_graph = self.ssr_module_graph(); + let InternalSsrChunkModule { ssr_module, .. } = *self.internal_ssr_chunk_module().await?; + Ok(trace_endpoint( + this.pages_project.project(), + Some(pages_function_name(&this.original_name).into()), + ssr_module_graph, + vec![*ssr_module], + )) + } } #[turbo_tasks::value] @@ -1734,6 +1749,11 @@ impl Endpoint for PageEndpoint { async fn project(self: Vc) -> Result> { Ok(self.await?.pages_project.project()) } + + #[turbo_tasks::function] + fn traced_files(self: Vc) -> Vc { + self.trace_result().all_files() + } } #[turbo_tasks::value] diff --git a/crates/next-api/src/route.rs b/crates/next-api/src/route.rs index 98557f117be4..4ea134e0959e 100644 --- a/crates/next-api/src/route.rs +++ b/crates/next-api/src/route.rs @@ -2,6 +2,7 @@ use std::fmt::Display; use anyhow::Result; use bincode::{Decode, Encode}; +use next_core::app_structure::FileSystemPathVec; use turbo_rcstr::RcStr; use turbo_tasks::{ Completion, FxIndexMap, FxIndexSet, NonLocalValue, OperationVc, ResolvedVc, TryFlatJoinIterExt, @@ -67,6 +68,12 @@ pub trait Endpoint { /// The project this endpoint belongs to. #[turbo_tasks::function] fn project(self: Vc) -> Vc; + + /// The traced files included by this endpoint. This is only used for analysis purposes. + /// Usually, `output()` includes the NFT file and everything else is handled outside of + /// Turbopack. + #[turbo_tasks::function] + fn traced_files(self: Vc) -> Vc; } #[derive( @@ -154,6 +161,15 @@ impl EndpointGroup { .collect(), ) } + + pub fn traced_files(&self) -> Vc { + traced_files_of_endpoints( + self.primary + .iter() + .map(|endpoint| *endpoint.endpoint) + .collect(), + ) + } } #[turbo_tasks::function] @@ -182,6 +198,17 @@ async fn module_graphs_of_endpoints( Ok(Vc::cell(module_graphs)) } +#[turbo_tasks::function] +async fn traced_files_of_endpoints( + endpoints: Vec>>, +) -> Result> { + let mut modules: FxIndexSet<_> = FxIndexSet::default(); + for endpoint in endpoints { + modules.extend(endpoint.traced_files().await?.iter().cloned()); + } + Ok(Vc::cell(modules.into_iter().collect())) +} + #[turbo_tasks::value(transparent)] pub struct EndpointGroups(Vec<(EndpointGroupKey, EndpointGroup)>); diff --git a/crates/next-core/Cargo.toml b/crates/next-core/Cargo.toml index 26cad2622132..a03c79e8a53e 100644 --- a/crates/next-core/Cargo.toml +++ b/crates/next-core/Cargo.toml @@ -87,6 +87,10 @@ turbopack-resolve = { workspace = true } turbopack-static = { workspace = true } turbopack-trace-utils = { workspace = true } +[dev-dependencies] +tokio = { workspace = true } +turbo-tasks-backend = { workspace = true } + [features] default = ["process_pool"] process_pool = ["turbopack-node/process_pool"] diff --git a/crates/next-core/src/next_config.rs b/crates/next-core/src/next_config.rs index c409d0331a6a..b75113053b8c 100644 --- a/crates/next-core/src/next_config.rs +++ b/crates/next-core/src/next_config.rs @@ -8,8 +8,8 @@ use serde_json::Value as JsonValue; use turbo_esregex::EsRegex; use turbo_rcstr::{RcStr, rcstr}; use turbo_tasks::{ - FxIndexMap, NonLocalValue, OperationValue, ResolvedVc, TaskInput, Vc, debug::ValueDebugFormat, - trace::TraceRawVcs, + FxIndexMap, NonLocalValue, OperationValue, ResolvedVc, TaskInput, TryJoinIterExt, Vc, + debug::ValueDebugFormat, trace::TraceRawVcs, }; use turbo_tasks_env::EnvMap; use turbo_tasks_fetch::FetchClientConfig; @@ -43,6 +43,7 @@ use crate::{ next_shared::{ transforms::ModularizeImportPackageConfig, webpack_rules::WebpackLoaderBuiltinCondition, }, + util::relativize_glob, }; #[turbo_tasks::value(transparent)] @@ -1694,6 +1695,53 @@ impl Issue for InvalidLoaderRuleConditionIssue { } } +#[turbo_tasks::value(transparent)] +pub struct OutputFileTracingIncludesExcludes( + #[bincode(with = "turbo_bincode::indexmap")] + FxIndexMap, Vec<(RcStr, FileSystemPath)>>, +); + +impl OutputFileTracingIncludesExcludes { + pub async fn parse( + project_path: FileSystemPath, + value: &Option, + ) -> Result { + if let Some(value) = value + && let Some(map) = value.as_object() + { + Ok(OutputFileTracingIncludesExcludes( + map.iter() + .map(async |(route_pattern, file_patterns)| { + let route_pattern = Glob::new( + RcStr::from(route_pattern.clone()), + GlobOptions { contains: true }, + ) + .to_resolved() + .await?; + let file_patterns = file_patterns + .as_array() + .iter() + .flat_map(|pattern| pattern.iter()) + .filter_map(|pattern| pattern.as_str()) + .map(async |pattern_str| { + let (glob, root) = relativize_glob(pattern_str, &project_path)?; + Ok((RcStr::from(glob), root)) + }) + .try_join() + .await?; + Ok((route_pattern, file_patterns)) + }) + .try_join() + .await? + .into_iter() + .collect(), + )) + } else { + Ok(OutputFileTracingIncludesExcludes(FxIndexMap::default())) + } + } +} + #[turbo_tasks::value_impl] impl NextConfig { #[turbo_tasks::function] @@ -2480,13 +2528,29 @@ impl NextConfig { } #[turbo_tasks::function] - pub fn output_file_tracing_includes(&self) -> Vc { - Vc::cell(self.output_file_tracing_includes.clone()) + pub async fn output_file_tracing_includes( + &self, + project_path: FileSystemPath, + ) -> Result> { + Ok(OutputFileTracingIncludesExcludes::parse( + project_path, + &self.output_file_tracing_includes, + ) + .await? + .cell()) } #[turbo_tasks::function] - pub fn output_file_tracing_excludes(&self) -> Vc { - Vc::cell(self.output_file_tracing_excludes.clone()) + pub async fn output_file_tracing_excludes( + &self, + project_path: FileSystemPath, + ) -> Result> { + Ok(OutputFileTracingIncludesExcludes::parse( + project_path, + &self.output_file_tracing_excludes, + ) + .await? + .cell()) } #[turbo_tasks::function] diff --git a/crates/next-core/src/util.rs b/crates/next-core/src/util.rs index e976abe84b5a..d81461b51733 100644 --- a/crates/next-core/src/util.rs +++ b/crates/next-core/src/util.rs @@ -1,4 +1,4 @@ -use std::{fmt::Display, str::FromStr}; +use std::{borrow::Cow, fmt::Display, str::FromStr}; use anyhow::{Result, bail}; use bincode::{Decode, Encode}; @@ -557,3 +557,208 @@ pub fn worker_forwarded_globals() -> Vec { rcstr!("NEXT_CLIENT_ASSET_SUFFIX"), ] } + +/// The globs defined in the next.config.mjs are relative to the project root. +/// The glob walker in turbopack is somewhat naive so we handle relative path directives first so +/// traversal doesn't need to consider them and can just traverse 'down' the tree. +/// The main alternative is to merge glob evaluation with directory traversal which is what the npm +/// `glob` package does, but this would be a substantial rewrite. +pub fn relativize_glob<'a>( + glob: &'a str, + relative_to: &FileSystemPath, +) -> Result<(&'a str, FileSystemPath)> { + let mut relative_to = Cow::Borrowed(relative_to); + let mut processed_glob = glob; + loop { + if let Some(stripped) = processed_glob.strip_prefix("../") { + if relative_to.path.is_empty() { + bail!( + "glob '{glob}' is invalid, it has a prefix that navigates out of the project \ + root" + ); + } + relative_to = Cow::Owned(relative_to.parent()); + processed_glob = stripped; + } else if let Some(stripped) = processed_glob.strip_prefix("./") { + processed_glob = stripped; + } else { + break; + } + } + Ok((processed_glob, relative_to.into_owned())) +} + +#[cfg(test)] +mod tests { + use turbo_tasks::ResolvedVc; + use turbo_tasks_backend::{BackendOptions, TurboTasksBackend, noop_backing_storage}; + use turbo_tasks_fs::{FileSystemPath, NullFileSystem}; + + use super::*; + + fn create_test_fs_path(path: &str) -> FileSystemPath { + FileSystemPath { + fs: ResolvedVc::upcast(NullFileSystem {}.resolved_cell()), + path: path.into(), + } + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_relativize_glob_normal_patterns() { + let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( + BackendOptions::default(), + noop_backing_storage(), + )); + tt.run_once(async { + // Test normal glob patterns without relative prefixes + let base_path = create_test_fs_path("project/src"); + + let (glob, path) = relativize_glob("*.js", &base_path).unwrap(); + assert_eq!(glob, "*.js"); + assert_eq!(path.path.as_str(), "project/src"); + + let (glob, path) = relativize_glob("components/**/*.tsx", &base_path).unwrap(); + assert_eq!(glob, "components/**/*.tsx"); + assert_eq!(path.path.as_str(), "project/src"); + + let (glob, path) = relativize_glob("lib/utils.ts", &base_path).unwrap(); + assert_eq!(glob, "lib/utils.ts"); + assert_eq!(path.path.as_str(), "project/src"); + Ok(()) + }) + .await + .unwrap(); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_relativize_glob_current_directory_prefix() { + let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( + BackendOptions::default(), + noop_backing_storage(), + )); + tt.run_once(async { + let base_path = create_test_fs_path("project/src"); + + // Single ./ prefix + let (glob, path) = relativize_glob("./components/*.tsx", &base_path).unwrap(); + assert_eq!(glob, "components/*.tsx"); + assert_eq!(path.path.as_str(), "project/src"); + + // Multiple ./ prefixes + let (glob, path) = relativize_glob("././utils.js", &base_path).unwrap(); + assert_eq!(glob, "utils.js"); + assert_eq!(path.path.as_str(), "project/src"); + + // ./ with complex glob + let (glob, path) = relativize_glob("./lib/**/*.{js,ts}", &base_path).unwrap(); + assert_eq!(glob, "lib/**/*.{js,ts}"); + assert_eq!(path.path.as_str(), "project/src"); + Ok(()) + }) + .await + .unwrap(); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_relativize_glob_parent_directory_navigation() { + let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( + BackendOptions::default(), + noop_backing_storage(), + )); + tt.run_once(async { + let base_path = create_test_fs_path("project/src/components"); + + // Single ../ prefix + let (glob, path) = relativize_glob("../utils/*.js", &base_path).unwrap(); + assert_eq!(glob, "utils/*.js"); + assert_eq!(path.path.as_str(), "project/src"); + + // Multiple ../ prefixes + let (glob, path) = relativize_glob("../../lib/*.ts", &base_path).unwrap(); + assert_eq!(glob, "lib/*.ts"); + assert_eq!(path.path.as_str(), "project"); + + // Complex navigation with glob + let (glob, path) = relativize_glob("../../../external/**/*.json", &base_path).unwrap(); + assert_eq!(glob, "external/**/*.json"); + assert_eq!(path.path.as_str(), ""); + Ok(()) + }) + .await + .unwrap(); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_relativize_glob_mixed_prefixes() { + let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( + BackendOptions::default(), + noop_backing_storage(), + )); + tt.run_once(async { + let base_path = create_test_fs_path("project/src/components"); + + // ../ followed by ./ + let (glob, path) = relativize_glob(".././utils/*.js", &base_path).unwrap(); + assert_eq!(glob, "utils/*.js"); + assert_eq!(path.path.as_str(), "project/src"); + + // ./ followed by ../ + let (glob, path) = relativize_glob("./../lib/*.ts", &base_path).unwrap(); + assert_eq!(glob, "lib/*.ts"); + assert_eq!(path.path.as_str(), "project/src"); + + // Multiple mixed prefixes + let (glob, path) = relativize_glob("././../.././external/*.json", &base_path).unwrap(); + assert_eq!(glob, "external/*.json"); + assert_eq!(path.path.as_str(), "project"); + Ok(()) + }) + .await + .unwrap(); + } + + #[tokio::test(flavor = "multi_thread", worker_threads = 2)] + async fn test_relativize_glob_error_navigation_out_of_root() { + let tt = turbo_tasks::TurboTasks::new(TurboTasksBackend::new( + BackendOptions::default(), + noop_backing_storage(), + )); + tt.run_once(async { + // Test navigating out of project root with empty path + let empty_path = create_test_fs_path(""); + let result = relativize_glob("../outside.js", &empty_path); + assert!(result.is_err()); + assert!( + result + .unwrap_err() + .to_string() + .contains("navigates out of the project root") + ); + + // Test navigating too far up from a shallow path + let shallow_path = create_test_fs_path("project"); + let result = relativize_glob("../../outside.js", &shallow_path); + assert!(result.is_err()); + assert!( + result + .unwrap_err() + .to_string() + .contains("navigates out of the project root") + ); + + // Test multiple ../ that would go out of root + let base_path = create_test_fs_path("a/b"); + let result = relativize_glob("../../../outside.js", &base_path); + assert!(result.is_err()); + assert!( + result + .unwrap_err() + .to_string() + .contains("navigates out of the project root") + ); + Ok(()) + }) + .await + .unwrap(); + } +}