Turbopack: don't crash the dev server when node_modules/next is briefly unresolvable (vercel#93877)

lukesandberg · web-flow · commit ceff72681ece · 2026-05-21T18:34:27.000Z
### What? When `node_modules/next` is briefly unresolvable mid-session (for example because pnpm is mid-install of a package with a `next` peer dependency, reshuffling its symlinks) Turbopack would: - Emit a `MissingNextFolderIssue` with severity `Fatal` - Propagate `anyhow!("Next.js package not found")` through ~10 levels of the build graph - Surface the failure at the napi boundary as a `TurbopackInternalError`, which the Next.js JS-side treats as fatal and shuts the dev server down The reported user impact was a dev session that became wedged and could not recover even after restarting `next dev`, because the persistent cache had stored the failed operation state. Users reported it as "stuck", "catastrophic", and required wiping `.next/` to recover. Unfortunately while i was able to reproduce a number of bad outcomes, i couldn't actually reproduce the issue of `next dev` getting stuck in a bad state. Still this PR will improve a number of apis and error messages See internal discussion: https://vercel.slack.com/archives/C046HAU4H7F/p1778792876550309 ### Why? A transient filesystem race during an install is the canonical recoverable error — once the install finishes, `next` is resolvable again. There is no reason this should bring down the dev server or poison the persistent cache. The bug had two contributing causes: 1. The error message was poor. It assumed the only cause was a mis-set `turbopack.root`, with the entire message in `title()`. It didn't acknowledge the install-race case or any of the other realistic causes (broken symlink, monorepo hoisting, removed `node_modules/next`, global install). Users had no way to know that the right response was to wait or refresh. 2. The Rust→napi boundary did not treat this as a recoverable error. Two specific code paths propagated `Err` instead of catching it: - `issue_filter_from_endpoint` in `endpoint.rs` calls `endpoint_op.connect().await?` — if the upstream endpoint resolution errors (as it does when `directory_tree_to_loader_tree` fails), this `?` propagates **before** the surrounding `strongly_consistent_catch_collectables` gets a chance to convert the failure into Issues. - `Project::hmr_version_state` is called bare from `project_hmr_events` at the napi boundary. There is no `_with_issues_operation` wrapper, so any error from the underlying `hmr_content`/`versioned_content_map.compute_entry` chain propagates straight to JS. ### How? Three changes: **1. Rewrite `MissingNextFolderIssue`** ([next_import_map.rs](crates/next-core/src/next_import_map.rs)) - Severity downgraded from `Fatal` to `Error`. `Fatal` triggers an explicit JS-side server shutdown in `print-build-errors.ts`; `Error` lets the dev server recover. - Title is a single line: `"Could not find the Next.js package (next/package.json)"`. - Description lists all realistic causes (concurrent install, missing/broken symlink, mis-set workspace root, monorepo hoisting, global install). - Doc link moved to `documentation_link()` so it doesn't pollute the inline message. **2. Make `issue_filter_from_endpoint` Err-tolerant** ([endpoint.rs](crates/next-napi-bindings/src/next_api/endpoint.rs)) - Wrap the `endpoint_op.connect().await` in a `match` and fall back to `IssueFilter::warnings_and_foreign_errors()` on `Err`. - This unblocks the existing `strongly_consistent_catch_collectables` recovery path, which then absorbs the upstream failure as Issues that surface through the napi boundary normally. **3. Add `hmr_version_state_with_issues_operation`** ([project.rs](crates/next-napi-bindings/src/next_api/project.rs)) - Mirrors the existing `hmr_update_with_issues_operation` pattern. - Drives `hmr_version_operation` as an `OperationVc`, catches any `Err` and falls back to `NotFoundVersion`, and collects issues from the operation via `peek_issues`. - The napi `project_hmr_events` handler now uses this wrapper instead of calling `Project::hmr_version_state` bare. Issues from both `hmr_version_state` and `hmr_update` are merged before being sent to JS. - Required lifting the inner `hmr_version_operation` out of `Project::hmr_version_state`'s closure to make it callable from the napi layer. A new regression test at [test/development/app-dir/concurrent-install/](test/development/app-dir/concurrent-install/) reproduces the failure by moving `node_modules/next` aside mid-HMR-session and asserts that the dev server does not emit `FATAL: An unexpected Turbopack error occurred` or `TurbopackInternalError`, and that the friendly Issue text surfaces. The test is skipped under `NEXT_SKIP_ISOLATE=1` since that mode doesn't produce a manipulable `node_modules`. Out of scope: after recovery, the dev server still returns 500 on subsequent requests until the user manually refreshes (the JS-side per-route error state isn't invalidated by Turbopack's successful recompile). That's a separate dev-server caching issue.
diff --git a/crates/next-core/src/next_import_map.rs b/crates/next-core/src/next_import_map.rs
@@ -1195,56 +1195,79 @@ impl Issue for MissingNextFolderIssue {
     }
 
     fn severity(&self) -> IssueSeverity {
-        IssueSeverity::Fatal
+        // In theory this should be fatal (how can we ever recover from next missing when we are
+        // next), but we actually might be detecting an ephemeral scenario where 'next' is moving
+        // and we can recover.
+        IssueSeverity::Error
     }
 
     fn stage(&self) -> IssueStage {
         IssueStage::Resolve
     }
 
     async fn title(&self) -> Result<StyledString> {
-        let system_path = match to_sys_path(self.path.clone()).await? {
-            Some(path) => path.to_str().unwrap_or("{unknown}").to_string(),
-            _ => "{unknown}".to_string(),
+        Ok(StyledString::Text(rcstr!(
+            "Could not find the Next.js package (next/package.json)"
+        )))
+    }
+
+    async fn description(&self) -> Result<Option<StyledString>> {
+        let context_path: RcStr = match to_sys_path(self.path.clone()).await? {
+            Some(path) => path.to_str().unwrap_or("{unknown}").into(),
+            _ => rcstr!("{unknown}"),
         };
-        let root_path = match to_sys_path(self.root.clone()).await? {
-            Some(path) => path.to_str().unwrap_or("{unknown}").to_string(),
-            _ => "{unknown}".to_string(),
+        let root_path: RcStr = match to_sys_path(self.root.clone()).await? {
+            Some(path) => path.to_str().unwrap_or("{unknown}").into(),
+            _ => rcstr!("{unknown}"),
         };
 
-        Ok(StyledString::Stack(vec![
-            StyledString::Line(vec![
-                StyledString::Text(
-                    "Error: Next.js inferred your workspace root, but it may not be correct."
-                        .into(),
-                ),
-            ]),
+        Ok(Some(StyledString::Stack(vec![
             StyledString::Line(vec![
-                StyledString::Text("We couldn't find the Next.js package (".into()),
-                StyledString::Strong("next/package.json".into()),
-                StyledString::Text(") from the project directory: ".into()),
-                StyledString::Strong(system_path.into()),
+                StyledString::Text(rcstr!("Resolved from: ")),
+                StyledString::Strong(context_path),
             ]),
             StyledString::Line(vec![
-                StyledString::Text("Filesystem root used for resolution: ".into()),
-                StyledString::Strong(root_path.into()),
+                StyledString::Text(rcstr!("Filesystem root used for resolution: ")),
+                StyledString::Strong(root_path),
             ]),
+            StyledString::Line(vec![StyledString::Text(rcstr!(""))]),
+            StyledString::Line(vec![StyledString::Text(rcstr!("Possible causes:"))]),
+            StyledString::Line(vec![StyledString::Text(rcstr!(
+                "  - node_modules is being reorganized by a concurrent install (e.g. pnpm adding \
+                 a package with a `next` peer dependency). This is transient and should clear \
+                 once the install completes."
+            ))]),
+            StyledString::Line(vec![StyledString::Text(rcstr!(
+                "  - node_modules/next was removed, renamed, or has a broken symlink."
+            ))]),
             StyledString::Line(vec![
-                StyledString::Text("To fix this, set ".into()),
-                StyledString::Code("turbopack.root".into()),
-                StyledString::Text(
-                    " in your Next.js config, or ensure the Next.js package is resolvable from the project directory.".into(),
-                ),
-            ]),
-            StyledString::Line(vec![
-                StyledString::Text("Note: For security and performance reasons, files outside of the project directory will not be compiled.".into()),
-            ]),
-            StyledString::Line(vec![
-                StyledString::Text("See ".into()),
-                StyledString::Strong("https://nextjs.org/docs/app/api-reference/config/next-config-js/turbopack#root-directory".into()),
-                StyledString::Text(" for more information.".into()),
+                StyledString::Text(rcstr!("  - The workspace root is incorrect — see ")),
+                StyledString::Code(rcstr!("turbopack.root")),
+                StyledString::Text(rcstr!(
+                    " in the Next.js config docs for how to configure it."
+                )),
             ]),
-        ]))
+            StyledString::Line(vec![StyledString::Text(rcstr!(
+                "  - In a monorepo, the Next.js package may only exist in a directory above the \
+                 closest directory containing a package manager lockfile. The workspace root is \
+                 detected by locating the nearest package manager lockfile."
+            ))]),
+            StyledString::Line(vec![StyledString::Text(rcstr!(
+                "  - Next.js is installed globally rather than as a project dependency. This is \
+                 not supported; install it locally."
+            ))]),
+            StyledString::Line(vec![StyledString::Text(rcstr!(""))]),
+            StyledString::Line(vec![StyledString::Text(rcstr!(
+                "Note: To ensure a hermetic build and a portable cache, files outside of the \
+                 workspace root are not compiled."
+            ))]),
+        ])))
+    }
+
+    fn documentation_link(&self) -> RcStr {
+        rcstr!(
+            "https://nextjs.org/docs/app/api-reference/config/next-config-js/turbopack#root-directory"
+        )
     }
 }
 
diff --git a/crates/next-napi-bindings/src/next_api/endpoint.rs b/crates/next-napi-bindings/src/next_api/endpoint.rs
@@ -101,14 +101,22 @@ impl Deref for ExternalEndpoint {
 
 /// Build an `IssueFilter` by reading the project from the endpoint's
 /// `OperationVc<OptionEndpoint>` and extracting ignore rules from its config.
-async fn issue_filter_from_endpoint(
-    endpoint_op: OperationVc<OptionEndpoint>,
-) -> Result<Vc<IssueFilter>> {
-    let endpoint_option = endpoint_op.connect().await?;
-    if let Some(ep) = &*endpoint_option {
-        Ok(ep.project().issue_filter())
-    } else {
-        Ok(IssueFilter::warnings_and_foreign_errors().cell())
+///
+/// If the upstream endpoint operation fails to resolve (e.g. because the build
+/// graph cannot be evaluated transiently — for example during a mid-session
+/// `node_modules` reshuffle), this falls back to a default filter rather than
+/// propagating the error.  In this scenario we believe the caller will already be observing the
+/// same error
+async fn issue_filter_from_endpoint(endpoint_op: OperationVc<OptionEndpoint>) -> Vc<IssueFilter> {
+    match endpoint_op.connect().await {
+        Ok(endpoint_option) => {
+            if let Some(ep) = &*endpoint_option {
+                ep.project().issue_filter()
+            } else {
+                IssueFilter::warnings_and_foreign_errors().cell()
+            }
+        }
+        Err(_) => IssueFilter::warnings_and_foreign_errors().cell(),
     }
 }
 
@@ -124,7 +132,7 @@ async fn get_written_endpoint_with_issues_operation(
     endpoint_op: OperationVc<OptionEndpoint>,
 ) -> Result<Vc<WrittenEndpointWithIssues>> {
     let write_to_disk_op = endpoint_write_to_disk_operation(endpoint_op);
-    let filter = issue_filter_from_endpoint(endpoint_op).await?;
+    let filter = issue_filter_from_endpoint(endpoint_op).await;
     let (written, issues, effects) =
         strongly_consistent_catch_collectables(write_to_disk_op, filter).await?;
     Ok(WrittenEndpointWithIssues {
@@ -229,25 +237,24 @@ async fn subscribe_issues_and_diags_operation(
 ) -> Result<Vc<EndpointIssuesAndDiags>> {
     let changed_op = endpoint_server_changed_operation(endpoint_op);
 
-    if should_include_issues {
-        let filter = issue_filter_from_endpoint(endpoint_op).await?;
-        let (changed_value, issues, effects) =
-            strongly_consistent_catch_collectables(changed_op, filter).await?;
-        Ok(EndpointIssuesAndDiags {
-            changed: changed_value,
-            issues,
-            effects,
-        }
-        .cell())
-    } else {
-        let changed_value = changed_op.read_strongly_consistent().await?;
-        Ok(EndpointIssuesAndDiags {
-            changed: Some(changed_value),
-            issues: Arc::new(vec![]),
-            effects: Arc::new(Effects::default()),
-        }
-        .cell())
+    // Use catch-collectables in both branches so transient build-graph errors
+    // (e.g. missing `node_modules/next` during a concurrent install) surface as
+    // Issues rather than killing the subscription with a `TurbopackInternalError`.
+    // When `should_include_issues` is false the caller doesn't need the Issue
+    // payload, but we still need the catch path to avoid the FATAL.
+    let filter = issue_filter_from_endpoint(endpoint_op).await;
+    let (changed_value, issues, effects) =
+        strongly_consistent_catch_collectables(changed_op, filter).await?;
+    Ok(EndpointIssuesAndDiags {
+        changed: changed_value,
+        issues: if should_include_issues {
+            issues
+        } else {
+            Arc::new(vec![])
+        },
+        effects,
     }
+    .cell())
 }
 
 #[tracing::instrument(level = "info", name = "get client-side endpoint changes", skip_all)]
@@ -265,11 +272,13 @@ pub fn endpoint_client_changed_subscribe(
             async move {
                 let changed_op = endpoint_client_changed_operation(endpoint_op);
                 // We don't capture issues and diagnostics here since we don't want to be
-                // notified when they change
+                // notified when they change.  We also want errors to propagate so we don't use
+                // strongly_consistent_catch_collectibles either.
                 //
                 // This must be a *read*, not just a resolve, because we need the root task created
                 // by `subscribe` to re-run when the `Completion`'s value changes (via equality),
                 // even if the cell id doesn't change.
+                //
                 let _ = changed_op.read_strongly_consistent().await?;
                 Ok(())
             }
diff --git a/crates/next-napi-bindings/src/next_api/project.rs b/crates/next-napi-bindings/src/next_api/project.rs
@@ -1821,6 +1821,10 @@ async fn hmr_update_with_issues_operation(
     target: HmrTarget,
 ) -> Result<Vc<HmrUpdateWithIssues>> {
     let update_op = project_hmr_update_operation(project, chunk_name, target, state);
+    // NOTE: we do not use `strongly_consistent_catch_collectables` here. The JS HMR
+    // consumers in `hot-reloader-turbopack.ts` (`subscribeToServerHmr` and
+    // `subscribeToClientHmrEvents`) rely on this read *throwing* on build-graph
+    // failures to trigger their recovery paths
     let update = update_op.read_strongly_consistent().await?;
     let filter = project.issue_filter();
     let issues = get_issues(update_op, filter).await?;
@@ -1957,6 +1961,12 @@ async fn get_hmr_chunk_names_with_issues_operation(
     target: HmrTarget,
 ) -> Result<Vc<HmrChunkNamesWithIssues>> {
     let hmr_chunk_names_op = project_hmr_chunk_names_operation(container, target);
+    // Do NOT switch this to `strongly_consistent_catch_collectables`. The JS HMR
+    // chunk-names consumer in `hot-reloader-turbopack.ts` relies on this read
+    // *throwing* on build-graph failures so its outer `try` block exits the
+    // subscription loop. Swallowing the error and emitting an empty chunk-name
+    // list keeps the loop running but with stale state, and obscures the real
+    // failure from the dev server log.
     let hmr_chunk_names = hmr_chunk_names_op.read_strongly_consistent().await?;
     let filter = issue_filter_from_container(container);
     let issues = get_issues(hmr_chunk_names_op, filter).await?;
diff --git a/packages/next/src/server/dev/turbopack-utils.ts b/packages/next/src/server/dev/turbopack-utils.ts
@@ -260,7 +260,7 @@ export async function handleRouteType({
           // otherwise we don't known when to unsubscribe and this leaking
           hooks?.subscribeToChanges(
             serverKey,
-            false,
+            /** includeIssues=*/ false,
             route.dataEndpoint,
             () => {
               // Report the next compilation again
@@ -279,7 +279,7 @@ export async function handleRouteType({
           )
           hooks?.subscribeToChanges(
             clientKey,
-            false,
+            /** includeIssues=*/ false,
             route.htmlEndpoint,
             () => {
               return {
@@ -296,7 +296,7 @@ export async function handleRouteType({
           if (entrypoints.global.document) {
             hooks?.subscribeToChanges(
               getEntryKey('pages', 'server', '_document'),
-              false,
+              /** includeIssues=*/ false,
               entrypoints.global.document,
               () => {
                 return {
@@ -354,7 +354,7 @@ export async function handleRouteType({
         // otherwise we don't known when to unsubscribe and this leaking
         hooks?.subscribeToChanges(
           key,
-          true,
+          /** includeIssues=*/ true,
           route.rscEndpoint,
           (change, hash) => {
             if (change.issues.some((issue) => issue.severity === 'error')) {
@@ -739,7 +739,7 @@ export async function handleEntrypoints({
     if (dev) {
       dev?.hooks.subscribeToChanges(
         key,
-        false,
+        /** includeIssues=*/ false,
         endpoint,
         async () => {
           const finishBuilding = dev.hooks.startBuilding(
@@ -876,7 +876,7 @@ export async function handlePagesErrorRoute({
     hooks.handleWrittenEndpoint(key, writtenEndpoint, false)
     hooks.subscribeToChanges(
       key,
-      false,
+      /** includeIssues=*/ false,
       entrypoints.global.app,
       () => {
         // There's a special case for this in `../client/page-bootstrap.ts`.
@@ -905,7 +905,7 @@ export async function handlePagesErrorRoute({
     hooks.handleWrittenEndpoint(key, writtenEndpoint, false)
     hooks.subscribeToChanges(
       key,
-      false,
+      /** includeIssues=*/ false,
       entrypoints.global.document,
       () => {
         return {
@@ -931,7 +931,7 @@ export async function handlePagesErrorRoute({
     hooks.handleWrittenEndpoint(key, writtenEndpoint, false)
     hooks.subscribeToChanges(
       key,
-      false,
+      /** includeIssues=*/ false,
       entrypoints.global.error,
       () => {
         // There's a special case for this in `../client/page-bootstrap.ts`.
diff --git a/test/development/app-dir/concurrent-install/app/late-route/page.tsx b/test/development/app-dir/concurrent-install/app/late-route/page.tsx
@@ -0,0 +1,3 @@
+export default function LateRoute() {
+  return <p>late route</p>
+}
diff --git a/test/development/app-dir/concurrent-install/app/layout.tsx b/test/development/app-dir/concurrent-install/app/layout.tsx
@@ -0,0 +1,8 @@
+import { ReactNode } from 'react'
+export default function Root({ children }: { children: ReactNode }) {
+  return (
+    <html>
+      <body>{children}</body>
+    </html>
+  )
+}
diff --git a/test/development/app-dir/concurrent-install/app/page.tsx b/test/development/app-dir/concurrent-install/app/page.tsx
@@ -0,0 +1,3 @@
+export default function Page() {
+  return <p>hello world</p>
+}
diff --git a/test/development/app-dir/concurrent-install/concurrent-install.test.ts b/test/development/app-dir/concurrent-install/concurrent-install.test.ts

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+export default function LateRoute() {`
	`2`	`+ return <p>late route</p>`
	`3`	`+}`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,3 @@`
	`1`	`+export default function Page() {`
	`2`	`+ return <p>hello world</p>`
	`3`	`+}`