Skip to content

runfiles: support Bazel 9 compact _repo_mapping format#921

Open
gregmagolan wants to merge 1 commit intomainfrom
compact_repo_mapping
Open

runfiles: support Bazel 9 compact _repo_mapping format#921
gregmagolan wants to merge 1 commit intomainfrom
compact_repo_mapping

Conversation

@gregmagolan
Copy link
Copy Markdown
Member

@gregmagolan gregmagolan commented Apr 6, 2026

Problem

Bazel 9 introduces --incompatible_compact_repo_mapping_manifest which rewrites _repo_mapping entries using a prefix-wildcard format to reduce file size from quadratic to linear in repo count. A source-repo field ending with * matches any canonical repo name that starts with that prefix.

The existing CSV parser only handled the old 3-column exact-match format. Any user on Bazel 9 with compact manifests enabled would get silently wrong runfile lookups for repos whose names match a compact prefix.

Solution

  • RepoMapping is promoted from a plain HashMap type alias to a struct holding separate exact and prefixed tables.
  • parse_repo_mapping detects compact entries via the trailing * (unambiguous: * is illegal in canonical repo names) and routes them to the prefix table.
  • RepoMapping::get tries exact match first, then iterates prefix entries in file order (first match wins).
  • New test test_parse_repo_mapping_compact_format covers mixed exact+prefix files, prefix matching, exact-over-prefix priority, and no-match cases.

References

  • bazelbuild/bazel#26262--incompatible_compact_repo_mapping_manifest tracking issue; contains the full format spec and the list of runfiles libraries that need updating
  • Bazel 9.0.0 release notes — compact format is enabled by default in Bazel 9

Test plan

  • bazel test //py/tools/runfiles/... passes on CI
  • Existing test_parse_repo_mapping and test_parse_repo_mapping_invalid_file still pass
  • New test_parse_repo_mapping_compact_format passes

Bazel 9 introduces --incompatible_compact_repo_mapping_manifest which
writes prefix-wildcard entries (source repo ending with `*`) to
_repo_mapping. Without this change, any repo whose canonical name
matches a compact prefix would silently fail to resolve runfiles.

The `*` suffix is unambiguous as a wildcard marker because it is
otherwise illegal in canonical repository names. Exact entries retain
priority over prefix entries; among prefix entries, the first match
wins (file order).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@gregmagolan gregmagolan force-pushed the compact_repo_mapping branch from c2598c8 to 95f778e Compare April 6, 2026 15:29
@aspect-workflows
Copy link
Copy Markdown

aspect-workflows bot commented Apr 6, 2026

Bazel 8 (Test)

11 test targets passed

Targets
//examples/pytest:main_with_colon_test [k8-fastbuild-ST-32abbbcff250]            605ms
//examples/pytest:pytest_test [k8-fastbuild-ST-32abbbcff250]                     2s
//py/private/py_venv:test_link [k8-fastbuild-ST-32abbbcff250]                    247ms
//py/tests/py-internal-venv:test [k8-fastbuild-ST-32abbbcff250]                  152ms
//py/tests/py-venv-disable-systemsite:test [k8-fastbuild-ST-32abbbcff250]        139ms
//py/tests/py-venv-disable-usersite:test [k8-fastbuild-ST-32abbbcff250]          106ms
//py/tests/py-venv-enable-site:test [k8-fastbuild-ST-32abbbcff250]               154ms
//py/tests/py-venv-standalone-interpreter:test [k8-fastbuild]                    72ms
//py/tests/py_venv_conflict:validate_import_roots [k8-fastbuild-ST-32abbbcff250] 322ms
//uv/private/gazelle_manifest:test [k8-fastbuild-ST-32abbbcff250]                1s
//uv/private/sdist_configure:detect_native_test [k8-fastbuild-ST-32abbbcff250]   428ms

Total test execution time was 6s. 100 tests (90.1%) were fully cached saving 50s.


Bazel 9 (Test)

9 test targets passed

Targets
//py/private/py_venv:test_link [k8-fastbuild-ST-32abbbcff250]                    326ms
//py/tests/py-internal-venv:test [k8-fastbuild-ST-32abbbcff250]                  252ms
//py/tests/py-venv-disable-systemsite:test [k8-fastbuild-ST-32abbbcff250]        307ms
//py/tests/py-venv-disable-usersite:test [k8-fastbuild-ST-32abbbcff250]          250ms
//py/tests/py-venv-enable-site:test [k8-fastbuild-ST-32abbbcff250]               211ms
//py/tests/py-venv-standalone-interpreter:test [k8-fastbuild]                    112ms
//py/tests/py_venv_conflict:validate_import_roots [k8-fastbuild-ST-32abbbcff250] 378ms
//uv/private/gazelle_manifest:test [k8-fastbuild-ST-32abbbcff250]                1s
//uv/private/sdist_configure:detect_native_test [k8-fastbuild-ST-32abbbcff250]   582ms

Total test execution time was 4s. 102 tests (91.9%) were fully cached saving 57s.


Bazel 8 (Test)

e2e

31 test targets passed

Targets
//cases/cross-repo-610:test [k8-fastbuild-ST-3d5e5c0e17c2]                       159ms
//cases/freethreaded-805:test [k8-fastbuild-ST-258424f943bf]                     5s
//cases/interpreter-features-836:test [k8-fastbuild]                             5s
//cases/interpreter-provisioning:test [k8-fastbuild-ST-1a8602e72ef0]             89ms
//cases/interpreter-version-541:test_3_10_venv [k8-fastbuild-ST-3746b67a8e25]    221ms
//cases/interpreter-version-541:test_3_11_venv [k8-fastbuild-ST-1a8602e72ef0]    109ms
//cases/interpreter-version-541:test_3_9_venv [k8-fastbuild-ST-cc09ec9be975]     223ms
//cases/pth-namespace-547:test [k8-fastbuild-ST-144eaf95864e]                    362ms
//cases/pytest-mock-530:test_mock_py_venv_test [k8-fastbuild-ST-ead18db1a9e5]    2s
//cases/repository-rule-deps-299:all_direct [k8-fastbuild-ST-1a8602e72ef0]       416ms
//cases/root-dir-paths-538:test_root_dir_venv [k8-fastbuild]                     120ms
//cases/uv-abi3-compat-853:test_import [k8-fastbuild-ST-8dad7e78101a]            177ms
//cases/uv-conflict-817:test_a [k8-fastbuild-ST-cd0c05bca563]                    465ms
//cases/uv-conflict-817:test_b [k8-fastbuild-ST-c24c68053f7a]                    580ms
//cases/uv-conflict-gte:test_a [k8-fastbuild-ST-feb2a29235ef]                    157ms
//cases/uv-conflict-gte:test_b [k8-fastbuild-ST-2677d4af01c2]                    102ms
//cases/uv-conflict-max-863:test_a [k8-fastbuild-ST-56c67bed1d4b]                135ms
//cases/uv-conflict-max-863:test_b [k8-fastbuild-ST-c1a5d65ffd6e]                136ms
//cases/uv-conflict-tilde-856:test_a [k8-fastbuild-ST-5d5cf6543ce5]              104ms
//cases/uv-conflict-tilde-856:test_b [k8-fastbuild-ST-12cade31d931]              240ms
//cases/uv-deps-650/airflow:airflow [k8-fastbuild-ST-032f9e1f9b8e]               223ms
//cases/uv-deps-650/extras:extras [k8-fastbuild-ST-d7def1929d69]                 585ms
//cases/uv-deps-650/say:say [k8-fastbuild-ST-3d5e5c0e17c2]                       324ms
//cases/uv-include-group:include_group [k8-fastbuild-ST-bb0a34f59f6f]            1s
//cases/uv-legacy-deps-750:googlemaps [k8-fastbuild-ST-9a8a2cef12dd]             808ms
//cases/uv-patching-829:test [k8-fastbuild-ST-2daf40585037]                      275ms
//cases/uv-plus-version:test [k8-fastbuild-ST-da4f8b55c8fe]                      789ms
//cases/uv-workspace-789:test [k8-fastbuild-ST-2f10d0569a36]                     802ms
//cases/venv-bin-scripts-423:test [k8-fastbuild-ST-7c878c5c0947]                 620ms
//cases/venv-internal-symlinks:test_internal_symlinks [k8-fastbuild-ST-2dc525242b17]640ms
//cases/venv-isolated-mode-703:test [k8-fastbuild-ST-7885c20ec43e]               588ms

Total test execution time was 22s. 20 tests (39.2%) were fully cached saving 14s.


Bazel 9 (Test)

e2e

30 test targets passed

Targets
//cases/cross-repo-610:test [k8-fastbuild-ST-3d5e5c0e17c2]                       149ms
//cases/freethreaded-805:test [k8-fastbuild-ST-258424f943bf]                     3s
//cases/interpreter-features-836:test [k8-fastbuild]                             7s
//cases/interpreter-provisioning:test [k8-fastbuild-ST-1a8602e72ef0]             178ms
//cases/interpreter-version-541:test_3_10_venv [k8-fastbuild-ST-3746b67a8e25]    232ms
//cases/interpreter-version-541:test_3_11_venv [k8-fastbuild-ST-1a8602e72ef0]    182ms
//cases/interpreter-version-541:test_3_9_venv [k8-fastbuild-ST-cc09ec9be975]     282ms
//cases/pth-namespace-547:test [k8-fastbuild-ST-144eaf95864e]                    340ms
//cases/pytest-mock-530:test_mock_py_venv_test [k8-fastbuild-ST-ead18db1a9e5]    3s
//cases/root-dir-paths-538:test_root_dir_venv [k8-fastbuild]                     183ms
//cases/uv-abi3-compat-853:test_import [k8-fastbuild-ST-8dad7e78101a]            263ms
//cases/uv-conflict-817:test_a [k8-fastbuild-ST-cd0c05bca563]                    639ms
//cases/uv-conflict-817:test_b [k8-fastbuild-ST-c24c68053f7a]                    631ms
//cases/uv-conflict-gte:test_a [k8-fastbuild-ST-feb2a29235ef]                    155ms
//cases/uv-conflict-gte:test_b [k8-fastbuild-ST-2677d4af01c2]                    218ms
//cases/uv-conflict-max-863:test_a [k8-fastbuild-ST-56c67bed1d4b]                197ms
//cases/uv-conflict-max-863:test_b [k8-fastbuild-ST-c1a5d65ffd6e]                164ms
//cases/uv-conflict-tilde-856:test_a [k8-fastbuild-ST-5d5cf6543ce5]              226ms
//cases/uv-conflict-tilde-856:test_b [k8-fastbuild-ST-12cade31d931]              132ms
//cases/uv-deps-650/airflow:airflow [k8-fastbuild-ST-032f9e1f9b8e]               163ms
//cases/uv-deps-650/extras:extras [k8-fastbuild-ST-d7def1929d69]                 640ms
//cases/uv-deps-650/say:say [k8-fastbuild-ST-3d5e5c0e17c2]                       214ms
//cases/uv-include-group:include_group [k8-fastbuild-ST-bb0a34f59f6f]            1s
//cases/uv-legacy-deps-750:googlemaps [k8-fastbuild-ST-9a8a2cef12dd]             674ms
//cases/uv-patching-829:test [k8-fastbuild-ST-2daf40585037]                      249ms
//cases/uv-plus-version:test [k8-fastbuild-ST-da4f8b55c8fe]                      755ms
//cases/uv-workspace-789:test [k8-fastbuild-ST-2f10d0569a36]                     1s
//cases/venv-bin-scripts-423:test [k8-fastbuild-ST-7c878c5c0947]                 1s
//cases/venv-internal-symlinks:test_internal_symlinks [k8-fastbuild-ST-2dc525242b17]372ms
//cases/venv-isolated-mode-703:test [k8-fastbuild-ST-7885c20ec43e]               441ms

Total test execution time was 24s. 21 tests (41.2%) were fully cached saving 17s.


Bazel 8 (Test)

examples/uv_pip_compile

All tests were cache hits

1 test (100.0%) was fully cached saving 444ms.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant