[Android][Testing] Run tests on supported Android API levels - #131809
Draft
matouskozak wants to merge 2 commits into
Draft
[Android][Testing] Run tests on supported Android API levels#131809matouskozak wants to merge 2 commits into
matouskozak wants to merge 2 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ec597b5d-622a-4f72-924d-b43a53c1613d
matouskozak
added
NO-MERGE
|
Azure Pipelines: Successfully started running 4 pipeline(s). 11 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
matouskozak
added
the
NO-MERGE
Contributor
|
Tagging subscribers to this area: @akoeplinger, @matouskozak, @simonrozsival |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adjusts Azure Pipelines + Helix queue selection to fan out Android x64 runtime and libraries testing across Android API levels 24–36, and updates pipeline triggering/timeout behavior to support the expanded test matrix.
Changes:
- Disabled PR triggering for the main
runtime.ymlpipeline. - Enabled PR triggering for
runtime-androidemulator.ymland expanded Android x64 Helix queue lists (API 24–36) for CoreCLR and Libraries templates. - Increased Android emulator job timeouts to accommodate longer end-to-end execution.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| eng/pipelines/runtime.yml | Disables PR triggers for the main runtime pipeline. |
| eng/pipelines/runtime-androidemulator.yml | Enables CI/PR triggers for the Android emulator wrapper pipeline. |
| eng/pipelines/libraries/helix-queues-setup.yml | Fans out Android x64 libraries testing across API 24–36 queues. |
| eng/pipelines/extra-platforms/runtime-extra-platforms-androidemulator.yml | Raises job timeouts for the expanded Android emulator test matrix. |
| eng/pipelines/coreclr/templates/helix-queues-setup.yml | Fans out Android x64 CoreCLR testing across API 24–36 queues (public). |
Comment on lines
+6
to
+13
| trigger: | ||
| branches: | ||
| include: | ||
| - release/*.* | ||
| paths: | ||
| include: | ||
| - '*' | ||
|
|
| - LICENSE.TXT | ||
| - PATENTS.TXT | ||
| - THIRD-PARTY-NOTICES.TXT | ||
| # Disabled for the Android API-level validation PR. |
Comment on lines
39
to
42
| nameSuffix: AllSubsets_NativeAOT_RuntimeTests | ||
| buildArgs: -s clr.alljits+clr.tools+clr.nativeaotruntime+clr.nativeaotlibs+libs -c $(_BuildConfig) | ||
| timeoutInMinutes: 240 | ||
| timeoutInMinutes: 1800 | ||
| # extra steps, run tests |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: ec597b5d-622a-4f72-924d-b43a53c1613d
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.
Suppressed comments (2)
eng/pipelines/runtime-androidemulator.yml:13
- The new CI
triggerwill run this very expensive pipeline on every push torelease/*.*(and on any path, including docs-only changes). The PR description only mentions enabling the PR trigger for validation; consider keeping CI disabled and only enablingpr:to avoid unexpected load on release branches.
trigger:
branches:
include:
- release/*.*
paths:
include:
- '*'
eng/pipelines/libraries/helix-queues-setup.yml:96
- The Android queue-selection comment is now out of date:
android_x64no longer uses a single Ubuntu Android queue, it fans out across API-specific queues (24-36). Please update the comment so it matches the logic.
# Android
# Use the Ubuntu-based Android queue for x86/x64/bionic_x64 on all projects,
# and also for arm/arm64/bionic_arm/bionic_arm64 on non-public projects (no internal Windows Android queue).
- ${{ if in(parameters.platform, 'android_x86', 'linux_bionic_x64') }}:
- Ubuntu.2204.Amd64.Android.29.Open
- ${{ if eq(parameters.platform, 'android_x64') }}:
- Ubuntu.2204.Amd64.Android.24.Open
Member
Author
|
/azp run runtime-androidemulator Note This command was posted by GitHub Copilot. |
|
No pipelines are associated with this pull request. |
Member
Author
|
/azp run runtime-androidemulator |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
Member
Author
|
Failure inventory for Android emulator build 1539422. Counts are visible by default; expand each API section for the individual failures and Helix console links. Wrapper rows are omitted when individual failed tests were published. Android API 24
Show 163 failed test records and 2 work-item-only failuresNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Note This failure inventory was generated by GitHub Copilot from Azure DevOps test results. |
Member
Author
|
Continuation of the failure inventory for Android emulator build 1539422. Counts are visible by default; expand each API section for details. Android API 25
Show 165 failed test records and 2 work-item-only failuresNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Note This failure inventory was generated by GitHub Copilot from Azure DevOps test results. |
Member
Author
|
Continuation of the failure inventory for Android emulator build 1539422. Counts are visible by default; expand each API section for details. Android API 26
Show 41 failed test records and 2 work-item-only failuresNativeAOT runtime
CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Android API 27
Show 57 failed test records and 9 work-item-only failuresNativeAOT runtime
CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Note This failure inventory was generated by GitHub Copilot from Azure DevOps test results. |
Member
Author
|
Continuation of the failure inventory for Android emulator build 1539422. Counts are visible by default; expand each API section for details. Android API 28
Show 168 failed test records and 3 work-item-only failuresNativeAOT runtime
CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Note This failure inventory was generated by GitHub Copilot from Azure DevOps test results. |
Member
Author
|
Continuation of the failure inventory for Android emulator build 1539422. Counts are visible by default; expand each API section for details. Android API 29
Show 2 failed test records and 0 work-item-only failuresNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Android API 30
Show 2 failed test records and 37 work-item-only failuresNativeAOT runtime
CoreCLR runtime
CoreCLR libraries
NativeAOT libraries
Note This failure inventory was generated by GitHub Copilot from Azure DevOps test results. |
Member
Author
|
Continuation of the failure inventory for Android emulator build 1539422. Counts are visible by default; expand each API section for details. Android API 31
Show 1 failed test record and 1 work-item-only failureNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Android API 32
Show 1 failed test record and 1 work-item-only failureNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Android API 33
Show 1 failed test record and 1 work-item-only failureNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Android API 34
Show 1 failed test record and 1 work-item-only failureNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Android API 35
Show 1 failed test record and 1 work-item-only failureNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Android API 36
Show 2 failed test records and 1 work-item-only failureNativeAOT runtime✅ No failures. CoreCLR runtime✅ No failures. CoreCLR libraries
NativeAOT libraries
Note This failure inventory was generated by GitHub Copilot from Azure DevOps test results. |
Member
Author
Android API-level test summaryBuild 1539422 exercised four legs on every API level: NativeAOT runtime, CoreCLR runtime, CoreCLR libraries, and NativeAOT libraries.
Main findings
Counts above exclude the synthetic Note This summary was generated by GitHub Copilot from Azure DevOps and Helix results. |
Member
Author
Highest-priority failures to investigate1. Android TLS cipher-suite parsing — APIs 24, 25, and 28Many HTTP/TLS failures on APIs 24 and 25 reduce to the Android TLS layer returning Failure logsAPI 24 System.Net.Http console API 28 AndroidPlatformTrust console 2. Dual-mode
|
Member
Author
|
I looked into the AVX NativeAOT issue and it looks like APIs 24/25 emulators do have AVX enabled yet 26/27 do not. This appears to be an combination of the virtualized host (linux VM) and the 26/27 android emulator defaults. Since the AVX tests passed on other levels, this doesn't seem to be a runtime but rather configuration issue. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run Android x64 runtime and library tests on API levels 24 through 36, following the approach used in #120093 and the minimum API-level update in #126838.
runtime-androidemulatorPR pipeline.runtime,runtime-coreclr hardware-intrinsics,runtime-diagnostics,runtime-dev-innerloop, anddotnet-linker-testsPR pipelines to avoid overloading CI.This is a test-only PR and must not be merged.
Summary is posted as a last comment
Note
This pull request was created with GitHub Copilot.