Add compiler source download integrity values #1749
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Eliminates the "canonical reproducible form" warnings that often appear for users that enable compiler dependency tracking. Fixes bazel-contrib#1743. Emits a detailed error message if the configured Scala version is unknown to rules_scala, and isn't provided by a `scala_deps.compiler_srcjar` object. This message provides guidance on how to create a suitable `scala_deps.compiler_srcjar` object. Details: - Adds `scripts/update_compiler_sources_integrity.py` to generate `scala/private/macros/compiler_sources_integrity.bzl`. Refactored code from `scripts/update_protoc_integrity.py` into `scripts/lib/update_integrity.py` for reuse by the new script. - Adds `test/shell/test_compiler_sources_integrity.sh` and adds it to `test_rules_scala.sh`. Adds the `test/compiler_sources_integrity` repo for use by the new test suite. The test suite is fast due to the test repo only containing an alias to `@scala_compiler_sources//:src`. - Fixes a problem with the regex in `run_tests` from `test_helper.sh` that prevented the test skipping mechanism from working. --- This eliminates an annoying warning for users, while providing improved build integrity. Win/win! The refactoring of common code into `scripts/lib` eliminated a lot of potential duplication. It's also exposed a common pattern for generating download integrity files for different artifacts.
Fixes breakages in build: - https://buildkite.com/bazel/rules-scala-scala/builds/5652 The lint fix is for `test/compiler_sources_integrity/BUILD`. The unbound variable error on macOS was in `test/shell/test_compiler_sources_integrity.sh`, even though the variable should've been bound. The script sets `-euo pipefail` anyway. Took the missing config setup from `test/shell/test_bzlmod_macros.sh` and added it to `test/shell/test_compiler_sources_integrity.sh`.
|
@simuons @liucijus As always, I'm happy to split this into separate pull requests, or at least separate commits, if you'd prefer. Either way, the recommended breakdown would be to review the parts in this order:
Also, @WojciechMazur, if this goes through, |
The build failed again with an unbound variable error on macOS: - https://buildkite.com/bazel/rules-scala-scala/builds/5653#019775ba-c5ff-4bff-b877-4bb22d22ec9b ``` running test test_emit_no_canonical_reproducible_form_warning_for_default_version Log: .../test/shell/test_compiler_sources_integrity.sh: line 85: build_args[@]: unbound variable build failed ``` Today I learned that older versions of Bash consider empty arrays unset, hence the application of `${build_args[@]+"${build_args[@]}"}`: - https://stackoverflow.com/a/7577209
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Eliminates the "canonical reproducible form" warnings that often appear for users that enable compiler dependency tracking. Fixes #1743.
Emits a detailed error message if the configured Scala version is unknown to rules_scala, and isn't provided by a
scala_deps.compiler_srcjarobject. This message provides guidance on how to create a suitablescala_deps.compiler_srcjarobject.Details:
Adds
scripts/update_compiler_sources_integrity.pyto generatescala/private/macros/compiler_sources_integrity.bzl. Refactored code fromscripts/update_protoc_integrity.pyintoscripts/lib/update_integrity.pyfor reuse by the new script.Adds
test/shell/test_compiler_sources_integrity.shand adds it totest_rules_scala.sh. Adds thetest/compiler_sources_integrityrepo for use by the new test suite. The test suite is fast due to the test repo only containing an alias to@scala_compiler_sources//:src.Fixes a problem with the regex in
run_testsfromtest_helper.shthat prevented the test skipping mechanism from working.Motivation
This eliminates an annoying warning for users, while providing improved build integrity. Win/win!
The refactoring of common code into
scripts/libeliminated a lot of potential duplication. It's also exposed a common pattern for generating download integrity files for different artifacts.