Skip to content

{AKS} Fix remaining CLI Runner test failures - #33856

Merged
Julie Zhu (yanzhudd) merged 3 commits into
Azure:devfrom
FumingZhang:fix/aks-live-test-remaining-failures
Aug 7, 2026
Merged

{AKS} Fix remaining CLI Runner test failures#33856
Julie Zhu (yanzhudd) merged 3 commits into
Azure:devfrom
FumingZhang:fix/aks-live-test-remaining-failures

Conversation

@FumingZhang

@FumingZhang FumingZhang commented Aug 6, 2026

Copy link
Copy Markdown
Member

🤖 PR Validation — ️✔️ All clear

Breaking Changes Tests
️✔️ None ️✔️ 130/130

Related command

az aks

Description

Fix the actionable failures observed in CLI Runner run 0c9345bc-24dd-493a-bab5-1ddce36bf31a:

  • pass the Container Insights solution JSON through a registered scenario kwarg so the live retry adapter does not interpret JSON braces as str.format placeholders;
  • initialize the Container Insights solution before the monitoring-addon MSI scenario creates its cluster, avoiding DCR creation before workspace tables exist;
  • wait for the cluster to settle between sequential VirtualMachines nodepool autoscaler operations, while keeping those additional requests out of cassette replay.

Artifact Streaming and Control Plane Metrics scenarios remain enabled. AKS RP investigation found that their failures reflect API-version and deployed-toggle rollout issues rather than deprecated features.

Testing Guide

AZURE_CONFIG_DIR=/tmp/clean-config VIRTUAL_ENV=/workspace/aenv \
  /workspace/aenv/bin/azdev test acs --no-exitfirst

Result: 980 passed, 84 skipped, 4 subtests passed.

VIRTUAL_ENV=/workspace/aenv /workspace/aenv/bin/azdev style acs

Result: pylint and flake8 passed.

The exact Python 3.12/3.14 CI failure was also reproduced and verified with:

python -m pytest -q \
  src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_provisioning_retry.py \
  src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_commands.py::AzureKubernetesServiceScenarioTest::test_aks_create_autoscaler_then_update_vms_pool

Result: 30 passed, 5 subtests passed.

History Notes

None. Test-only changes.


This checklist is used to make sure that common guidelines for a pull request are followed.

Avoid retry formatting of inline JSON, initialize Container Insights workspaces before AKS creation, wait between sequential VMS pool operations, and skip live scenarios whose service toggles are unavailable in the runner subscription.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@yonzhan

Copy link
Copy Markdown
Collaborator

AKS

@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

Artifact Streaming is GA and Control Plane Metrics is on the GA rollout path, so retain their live coverage instead of masking API-version and toggle-rollout failures.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@FumingZhang
FumingZhang marked this pull request as ready for review August 6, 2026 06:39
@FumingZhang
FumingZhang requested a review from a team as a code owner August 6, 2026 06:39
Copilot AI lite review requested due to automatic review settings August 6, 2026 06:39
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@FumingZhang

Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates AKS scenario tests to address remaining CLI Runner live-test failures by avoiding accidental str.format interpolation of inline JSON, ensuring prerequisite Container Insights resources exist before cluster creation, and adding waits to reduce flakiness between sequential nodepool autoscaler operations.

Changes:

  • Pass the Container Insights solution JSON via a registered kwargs placeholder ({container_insights_solution}) instead of embedding JSON directly in the formatted command string.
  • Create the Log Analytics workspace/solution before running the MSI monitoring-addon cluster create, and pass --workspace-resource-id.
  • Add az aks wait --updated calls between sequential VirtualMachines autoscaler/nodepool operations.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_provisioning_retry.py Adds a unit test asserting Container Insights solution JSON is passed via a registered scenario kwarg (not inlined into the command string).
src/azure-cli/azure/cli/command_modules/acs/tests/latest/test_aks_commands.py Updates live scenario logic to use kwarg-based solution JSON, pre-create workspace for monitoring-addon MSI scenario, and adds waits between autoscaler operations.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Gate the VMS autoscaler settle checks to live and recording runs so existing replay cassettes do not issue unrecorded GET requests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@a0x1ab

Copy link
Copy Markdown
Member

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 3 pipeline(s).

@yanzhudd
Julie Zhu (yanzhudd) merged commit 716b3a3 into Azure:dev Aug 7, 2026
50 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

act-observability-squad AKS az aks/acs/openshift Auto-Assign Auto assign by bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants