False-positive cache busts from over-broad dependency "bleedthrough"

[bitttttten]

Summary

Turborepo’s build graph can sometimes be too permissive for dev-only packages, causing false-positive cache invalidations.

With the "basic" example, we can see we have the web app and the @repo/eslint-config package.

Given pnpm exec turbo build --filter=web --dry=json, we see:

{
  // ..
  "tasks": [
    {
      "taskId": "@repo/eslint-config#build",
      "task": "build",
      "package": "@repo/eslint-config",
      "hash": "7f189443198f6183",
      "inputs": {
        "README.md": "8b42d901b0e8dee2e21313ec0442c50e9fef038a",
        "base.js": "09d316efdb6f6db4f6141815e87980a598f6ff54",
        "next.js": "4df088ac8ddeca380f3568f74778906a90314d0b",
        "package.json": "5f1f5ff4f4cbb204d4907b9768174e2e1c190945",
        "react-internal.js": "daeccba24d4f4b95ad293d175b570acd126628a9"
      },
      /// ...
    },
    {
      "taskId": "@repo/typescript-config#build",
      "task": "build",
      "package": "@repo/typescript-config",
      "hash": "d1a04ccd0868d600",
      "inputs": {
        "base.json": "5117f2a3d1c5fe54a344a7152acbe366fe63cdda",
        "nextjs.json": "e6defa48fce860cf5570f268d4880161a74789c8",
        "package.json": "27c0e60436aac79bd14661e016c8c5721c5db6d6",
        "react-library.json": "c3a1b26fbb3b6ad5d606836247a8ca3a1be051c6"
      },
      // ..
    } 
  ]    
}

We can see the build for web app relies on all the build steps of all it's dependencies. Since the build step is configured on the root, in $TURBO_ROOT$/turbo.json, it picks up the $TURBO_DEFAULT$ files. For example, if we edit

Why it happens: Because build depends on ^build and web lists @repo/eslint-config as a devDependency, Turbo treats @repo/eslint-config#build as an upstream task. With $TURBO_DEFAULT$ inputs, Turbo hashes every file in that package (including README.md, base.js, next.js). Any change there updates the upstream task hash, which bubbles to web#build even though ESLint config changes don’t affect the runtime build output.

The impact: Non-material edits (docs or lint rule files) invalidate the web build cache, leading to unnecessary rebuilds and misleading cache misses.

Repro: pnpm exec turbo build --filter=web --dry=json shows @repo/eslint-config#build in the graph with inputs that include README.md and ESLint rule files. You can also run the web build until you see cache hit. Now edit the base.js file (no direct impact on build), or the eslint-config README (also not impact), then re-run the web build and see cache miss.

My thoughts

• We cannot amend ^build with a tighter dependency rule, as we don't know what the files the web app is importing from.
• We could adjust ^build to only include files in the build step if the package has a build script? i.e. omitting eslint-configs from being a part of the build dependency graph. However, this also makes no sense as we don't know how the file is imported.
• Sprinkle turbo.json files all over the place and customise the build step per package. i.e. for eslint-config we could do "inputs": []. This is a weird dev exp, and it also means tons of turbo.json files sprinkled all over the place. Would you also want a warning when an app is importing from another package with no inputs? What I mean is now we open ourselves to the cache being too strict, if a developer now imports from eslint-config as part of the build, they'd want the build cache to invalidate as they'd assume it would.

Thoughts

I am happy to manage turbo.json files all over the place for now, just mainly opening this up for discussion. Perhaps there are native solutions I'm missing, or members of the community have solved this already.

Looking forward to hearing what people say!

[anthonyshew]

I don't really understand what is being asked here. The default behavior of using the hash of a package as a whole is a safe default, and, if you want to tune the behavior, you absolutely can using inputs and Package Configurations as is described.

While I agree it would be great to be able to make Turborepo have fancy inputs by default that e.g. ignore README.md, such behavior would be incredibly difficult to educate and learn about for simple, basic usage of Turborepo. For advanced users, inputs is the way to tune tasks for maximizing performance.

[bitttttten]

I guess my ask is "What is recommended?" 👻

We have a bunch of packages (just to give some concrete examples) like "test-scripts" or "aws-scripts" or "eslint-configs" (like the example mentioned) which are bleeding into our build cache. We see that they don't impact the build directly, yet bleed over.

Then, two thoughts come from it: maybe there is a built in way for turbo to manage this OR is there a recommended way to handle it?

As I understand, you are saying to fine tune it, just handle the inputs yourself, which makes sense 🫡

I can totally imagine that analysing the dep graph in a way of marking exactly which files were used and include them in the build is maybe an impossible task.. I just never know with turbo 😉

[bitttttten]

Feel free to close it if my reply makes sense to you.

[anthonyshew]

I see. Yes, you'll want to tune this behavior for your needs today - but also we can make Turborepo smarter in the future.