Skip to content

Test Case 1: All Green - #878

Open
oxve wants to merge 16 commits into
mainfrom
test-case-1-all-green
Open

Test Case 1: All Green#878
oxve wants to merge 16 commits into
mainfrom
test-case-1-all-green

Conversation

@oxve

@oxve oxve commented Aug 28, 2026

Copy link
Copy Markdown
Member

Verifying all green path for CI Shepherd.

Bug: 551996294

oxve added 7 commits August 28, 2026 16:45
- Update junit_mini_parser.py to output failing tests as structured JSON.
- Update junit_mini_parser_test.py with full test coverage and typing annotations.
- Update main.yaml and process_test_results to parse JSON output using jq.

Tag: agy
Conv: fb32e968-5c28-4711-9b18-47b6c44fd1d0
Update process_test_results action to use set -euo pipefail and improve quoting and failure output parsing.

Tag: agy
Conv: 497490ac-8ae1-409a-88cb-30bc08e7ffe6
Bug: 551996294
Remove error handling and empty checks from junit_mini_parser.py.
Simplify test_failures.json parsing in process_test_results and main.yaml workflows to use a unified jq query without empty checks.

Tag: agy
Conv: ef023ce0-0476-4a39-82ad-153794531729
Bug: 546722232
Restore the command-line arguments check and logging configuration in the entry point of junit_mini_parser.py as requested.

Tag: agy
Conv: ef023ce0-0476-4a39-82ad-153794531729
Bug: 546722232
Update process_test_results and main.yaml to check if the first XML file exists before calling junit_mini_parser.py.
This prevents crashes when no XML files match the glob and nullglob is not enabled.

Tag: agy
Conv: ef023ce0-0476-4a39-82ad-153794531729
Bug: 546722232
…evice tests

Implement test retry logic in GitHub Actions for on-host and on-device
platforms. This introduces a mechanism to track and retry specific
failures in PRs while allowing full retries on push events.

Update test filtering and reporting to support automated generation of retry
filters and consolidated test reporting. This reduces manual developer
intervention for flaky tests and optimizes data uploads to external
monitoring tools by merging result sets.

Tag: agy
Conv: b0702142-38ef-440d-acc8-a87a139ea206
Bug: 546722232
Move test_filter.py and test_filter_test.py to cobalt/devinfra/github.
Update references in main.yaml, on_host_tests, and on_device_tests workflows.
Simplify generate_retry_filter action by removing unnecessary empty check.

Tag: agy
Conv: ef023ce0-0476-4a39-82ad-153794531729
Bug: 546722232
@github-actions

Copy link
Copy Markdown
Contributor

🤖 Gemini Suggested Commit Message


ci: Mock build and test steps for CI Shepherd

Replace standard build and test execution steps in GitHub Actions with
mocked versions to facilitate CI Shepherd integration testing. This
allows for a fast and predictable "all green" path through the CI
pipeline without the overhead of full compilation or actual test
execution.

The update introduces a dedicated Python script to simulate job success
or failure and generates dummy artifacts to satisfy workflow
dependencies.

Bug: None

💡 Pro Tips for a Better Commit Message:

  1. Influence the Result: Want to change the output? You can write custom prompts or instructions directly in the Pull Request description. The model uses that text to generate the message.
  2. Re-run the Generator: Post a comment with: /generate-commit-message

@github-actions

github-actions Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor
Context State Description
CI / android ✅ PASS Passed
CI / linux ✅ PASS Passed

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request mocks several GitHub Actions workflow steps—including GN generation, Ninja builds, browser tests, on-host tests, and artifact archiving—by replacing actual execution commands with a mock script (mock_step.py) and dummy artifact generation. The review feedback focuses on improving the robustness and portability of these mock implementations. Specifically, it suggests locating the configuration file relative to the script's path, handling relative paths safely in os.makedirs, using os.path.splitext to correctly parse filenames with multiple dots, and piping tar output to zstd to ensure cross-platform compatibility on macOS runners.

Comment thread .github/scripts/mock_step.py Outdated
return parser.parse_args()

def load_config():
config_path = os.path.join(os.environ.get('GITHUB_WORKSPACE', '.'), '.github/config/mock_config.json')

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Relying on GITHUB_WORKSPACE or the current working directory . makes the script less portable and harder to run or test locally. It is more robust to locate the configuration file relative to the script's own path using __file__.

    script_dir = os.path.dirname(os.path.abspath(__file__))
    config_path = os.path.abspath(os.path.join(script_dir, '..', 'config', 'mock_config.json'))

Comment thread .github/scripts/mock_step.py Outdated
failure.text = 'Mocked failure details'

tree = ET.ElementTree(root)
os.makedirs(os.path.dirname(filepath), exist_ok=True)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If filepath is a relative path with no directory component (e.g., just a filename), os.path.dirname(filepath) will return an empty string. Calling os.makedirs('', exist_ok=True) will raise a FileNotFoundError. It is safer to check if the directory path is non-empty before calling os.makedirs.

Suggested change
os.makedirs(os.path.dirname(filepath), exist_ok=True)
dirname = os.path.dirname(filepath)
if dirname:
os.makedirs(dirname, exist_ok=True)

Comment thread .github/scripts/mock_step.py Outdated
for target_path in targets:
filename = os.path.basename(target_path)
# Remove extension if any (like .exe or run_ prefix)
test_name = filename.split('.')[0]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using filename.split('.')[0] to strip the file extension will incorrectly truncate filenames that contain multiple dots (e.g., foo.bar.test). Using os.path.splitext is the standard and robust way to separate a file's base name from its extension.

Suggested change
test_name = filename.split('.')[0]
test_name = os.path.splitext(filename)[0]

Comment on lines +33 to +34
tar --zstd -cf "${GITHUB_WORKSPACE}/artifacts/test_artifacts.tar.zstd" dummy.txt
tar --zstd -cf "${GITHUB_WORKSPACE}/artifacts/cobalt_browsertests_deps.tar.zstd" dummy.txt

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using tar --zstd is specific to GNU tar and may fail on macOS runners (which use BSD tar by default). To ensure cross-platform compatibility across different runner operating systems, you can pipe the tar output to zstd directly.

        tar -cf - dummy.txt | zstd -z > "${GITHUB_WORKSPACE}/artifacts/test_artifacts.tar.zstd"
        tar -cf - dummy.txt | zstd -z > "${GITHUB_WORKSPACE}/artifacts/cobalt_browsertests_deps.tar.zstd"

oxve added 5 commits August 31, 2026 18:41
- Add cobalt/devinfra to CI_ESSENTIALS in main.yaml.
- Rename validation rollup jobs to ${{ matrix.name }}_validation and tvos_validation to avoid erroneous retries from shepherd/deflake workflows.
- Fix bash parameter expansion for test_failures in main.yaml validation step.
- Sanitize colon prefixes in test target names across test_filter.py and actions.
- Support dictionary-formatted and string-formatted test targets in on_device_tests and on_host_tests actions and on_device_tests_gateway_client.py.
- Update test_filter_test.py with coverage for colon-prefixed target names.

Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 546722232
Tag: agy
Conv: 6ba17f35-7b52-48e7-a275-931ef58d3366
Bug: 546722232
Fix formatting violation in cobalt/devinfra/github/test_filter_test.py
to satisfy pre-commit yapf hook check.

Tag: agy
Conv: f5adcc09-6b58-4e6a-9594-32e4a55670ba
Bug: 546722232
…ion reporting

Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 551996294
Add mock_step.py and actions/workflow mocking configuration to execute simulated fast builds and tests on hosted runners for CI Shepherd verification.

Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 551996294
@oxve
oxve force-pushed the test-case-1-all-green branch from cfd50e6 to 1919e2f Compare August 31, 2026 19:28
oxve added 3 commits August 31, 2026 19:48
Mock .github/actions/checkout and .github/actions/depot_tools to skip multi-gigabyte repo cloning and tool bootstrapping in sandbox verification testing.

Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 551996294
Configure deflake.json with deflake_runs=2 and fail test on shard 1 through attempt 2 to verify custom max attempts limit.

Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 551996294
Ensure checkout action copies CI_ESSENTIALS into cobalt/src and build action populates test targets and dummy APK artifacts without network calls.

Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 551996294
Tag: agy
Conv: a62819b8-a42e-437a-b0f5-972911759cd8
Bug: 551996294
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant