Test Case 1: All Green - #878
Conversation
- Update junit_mini_parser.py to output failing tests as structured JSON. - Update junit_mini_parser_test.py with full test coverage and typing annotations. - Update main.yaml and process_test_results to parse JSON output using jq. Tag: agy Conv: fb32e968-5c28-4711-9b18-47b6c44fd1d0
Update process_test_results action to use set -euo pipefail and improve quoting and failure output parsing. Tag: agy Conv: 497490ac-8ae1-409a-88cb-30bc08e7ffe6 Bug: 551996294
Remove error handling and empty checks from junit_mini_parser.py. Simplify test_failures.json parsing in process_test_results and main.yaml workflows to use a unified jq query without empty checks. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
Restore the command-line arguments check and logging configuration in the entry point of junit_mini_parser.py as requested. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
Update process_test_results and main.yaml to check if the first XML file exists before calling junit_mini_parser.py. This prevents crashes when no XML files match the glob and nullglob is not enabled. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
…evice tests Implement test retry logic in GitHub Actions for on-host and on-device platforms. This introduces a mechanism to track and retry specific failures in PRs while allowing full retries on push events. Update test filtering and reporting to support automated generation of retry filters and consolidated test reporting. This reduces manual developer intervention for flaky tests and optimizes data uploads to external monitoring tools by merging result sets. Tag: agy Conv: b0702142-38ef-440d-acc8-a87a139ea206 Bug: 546722232
Move test_filter.py and test_filter_test.py to cobalt/devinfra/github. Update references in main.yaml, on_host_tests, and on_device_tests workflows. Simplify generate_retry_filter action by removing unnecessary empty check. Tag: agy Conv: ef023ce0-0476-4a39-82ad-153794531729 Bug: 546722232
🤖 Gemini Suggested Commit Message💡 Pro Tips for a Better Commit Message:
|
|
There was a problem hiding this comment.
Code Review
This pull request mocks several GitHub Actions workflow steps—including GN generation, Ninja builds, browser tests, on-host tests, and artifact archiving—by replacing actual execution commands with a mock script (mock_step.py) and dummy artifact generation. The review feedback focuses on improving the robustness and portability of these mock implementations. Specifically, it suggests locating the configuration file relative to the script's path, handling relative paths safely in os.makedirs, using os.path.splitext to correctly parse filenames with multiple dots, and piping tar output to zstd to ensure cross-platform compatibility on macOS runners.
| return parser.parse_args() | ||
|
|
||
| def load_config(): | ||
| config_path = os.path.join(os.environ.get('GITHUB_WORKSPACE', '.'), '.github/config/mock_config.json') |
There was a problem hiding this comment.
Relying on GITHUB_WORKSPACE or the current working directory . makes the script less portable and harder to run or test locally. It is more robust to locate the configuration file relative to the script's own path using __file__.
script_dir = os.path.dirname(os.path.abspath(__file__))
config_path = os.path.abspath(os.path.join(script_dir, '..', 'config', 'mock_config.json'))| failure.text = 'Mocked failure details' | ||
|
|
||
| tree = ET.ElementTree(root) | ||
| os.makedirs(os.path.dirname(filepath), exist_ok=True) |
There was a problem hiding this comment.
If filepath is a relative path with no directory component (e.g., just a filename), os.path.dirname(filepath) will return an empty string. Calling os.makedirs('', exist_ok=True) will raise a FileNotFoundError. It is safer to check if the directory path is non-empty before calling os.makedirs.
| os.makedirs(os.path.dirname(filepath), exist_ok=True) | |
| dirname = os.path.dirname(filepath) | |
| if dirname: | |
| os.makedirs(dirname, exist_ok=True) |
| for target_path in targets: | ||
| filename = os.path.basename(target_path) | ||
| # Remove extension if any (like .exe or run_ prefix) | ||
| test_name = filename.split('.')[0] |
There was a problem hiding this comment.
Using filename.split('.')[0] to strip the file extension will incorrectly truncate filenames that contain multiple dots (e.g., foo.bar.test). Using os.path.splitext is the standard and robust way to separate a file's base name from its extension.
| test_name = filename.split('.')[0] | |
| test_name = os.path.splitext(filename)[0] |
| tar --zstd -cf "${GITHUB_WORKSPACE}/artifacts/test_artifacts.tar.zstd" dummy.txt | ||
| tar --zstd -cf "${GITHUB_WORKSPACE}/artifacts/cobalt_browsertests_deps.tar.zstd" dummy.txt |
There was a problem hiding this comment.
Using tar --zstd is specific to GNU tar and may fail on macOS runners (which use BSD tar by default). To ensure cross-platform compatibility across different runner operating systems, you can pipe the tar output to zstd directly.
tar -cf - dummy.txt | zstd -z > "${GITHUB_WORKSPACE}/artifacts/test_artifacts.tar.zstd"
tar -cf - dummy.txt | zstd -z > "${GITHUB_WORKSPACE}/artifacts/cobalt_browsertests_deps.tar.zstd"b400838 to
9ac7e7c
Compare
- Add cobalt/devinfra to CI_ESSENTIALS in main.yaml.
- Rename validation rollup jobs to ${{ matrix.name }}_validation and tvos_validation to avoid erroneous retries from shepherd/deflake workflows.
- Fix bash parameter expansion for test_failures in main.yaml validation step.
- Sanitize colon prefixes in test target names across test_filter.py and actions.
- Support dictionary-formatted and string-formatted test targets in on_device_tests and on_host_tests actions and on_device_tests_gateway_client.py.
- Update test_filter_test.py with coverage for colon-prefixed target names.
Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 546722232
Tag: agy Conv: 6ba17f35-7b52-48e7-a275-931ef58d3366 Bug: 546722232
Fix formatting violation in cobalt/devinfra/github/test_filter_test.py to satisfy pre-commit yapf hook check. Tag: agy Conv: f5adcc09-6b58-4e6a-9594-32e4a55670ba Bug: 546722232
…ion reporting Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Add mock_step.py and actions/workflow mocking configuration to execute simulated fast builds and tests on hosted runners for CI Shepherd verification. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
cfd50e6 to
1919e2f
Compare
Mock .github/actions/checkout and .github/actions/depot_tools to skip multi-gigabyte repo cloning and tool bootstrapping in sandbox verification testing. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Configure deflake.json with deflake_runs=2 and fail test on shard 1 through attempt 2 to verify custom max attempts limit. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Ensure checkout action copies CI_ESSENTIALS into cobalt/src and build action populates test targets and dummy APK artifacts without network calls. Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
e69c706 to
07dac0d
Compare
Tag: agy Conv: a62819b8-a42e-437a-b0f5-972911759cd8 Bug: 551996294
Verifying all green path for CI Shepherd.
Bug: 551996294