Skip to content

[test-improver] Add unit tests for RetryExtensions.AddRetryProvider #11397

Description

@nohwnd-bot

Goal and rationale

RetryExtensions.AddRetryProvider is the single public wiring entry point for the entire retry-failed-tests feature — it registers the command-line options provider, the test-host application lifetime callback, the composite data-consumer/session-lifetime handler, and the test-host orchestrator. No existing Retry-area test file (RetryTests, RetryDataConsumerTests, RetryArgumentsBuilderTests, RetryOrchestratorHelperTests) exercised this method directly — each one constructs its individual extension class in isolation, so a regression that broke the wiring itself (e.g. forgetting to register one piece, or wiring the pieces to different instances) would go undetected.

Approach

Added RetryExtensionsTests.cs with 3 tests calling AddRetryProvider on a real ITestApplicationBuilder and then driving the platform's own build pipeline to verify the registrations:

  • Command-line options provider discoverable via the internal CommandLineManager's factory list.
  • Exactly one test-host-orchestrator factory registered, and it builds a RetryOrchestrator.
  • The lifecycle callback, data consumer, and test session lifetime handler all resolve to the same RetryDataConsumer instance (mirrors the real TestHostBuilder pipeline order: lifecycle callback built and registered into the ServiceProvider first, then the composite data-consumer/session-lifetime-handler factory is built with a single shared alreadyBuiltServices list across both calls).

Coverage impact

Before After
Microsoft.Testing.Extensions.UnitTests (net8.0) total 1876 1879
Failed 0 0

Trade-offs

Uses reflection on the internal CommandLineManager._commandLineProviderFactory field to avoid needing to construct a full IConfiguration just to call CommandLineManager.BuildAsync; this is a bit more coupled to the implementation than an end-to-end call, but keeps the test focused on what AddRetryProvider is responsible for.

Reproducibility

export PATH="$PWD/.dotnet:$PATH"
dotnet build test/UnitTests/Microsoft.Testing.Extensions.UnitTests/Microsoft.Testing.Extensions.UnitTests.csproj -c Debug -f net8.0
./artifacts/bin/Microsoft.Testing.Extensions.UnitTests/Debug/net8.0/Microsoft.Testing.Extensions.UnitTests --filter "FullyQualifiedName~RetryExtensionsTests"

Test Status

Build succeeded (0 warnings/errors) for both net8.0 and net9.0. Targeted suite 3/3 passed. Full Microsoft.Testing.Extensions.UnitTests net8.0 suite: 1879 total, 0 failed, 37 skipped (pre-existing, no regressions). dotnet format whitespace TestFx.slnx --verify-no-changes --include <file> clean (only the expected harmless F#-project warning).


Note

This was originally intended as a pull request, but the git push operation failed.

Original error: The process '/usr/bin/git' failed with exit code 128

Workflow Run: View run details and download bundle artifact

The bundle file is available in the agent artifact in the workflow run linked above.

Create the pull request manually
# Download the artifact from the workflow run
gh run download '35543089018' -n agent -D '/tmp/agent-35543089018'

# Resolve the bundle source ref, fetch it into a temporary ref, then create the local branch
bundle_path='/tmp/agent-35543089018/aw-test-assist-retry-extensions-tests-v2.bundle'
temp_ref='refs/bundles/create-pr-test-assist-retry-extensions-tests-v2-9c47722bf5a10e3f-f7d1c4d8'
target_ref='refs/heads/test-assist/retry-extensions-tests-v2-9c47722bf5a10e3f'
bundle_source_ref=$(git bundle list-heads "$bundle_path" | awk '$2 ~ /^refs\/heads\// { print $2 }')
if [ -z "$bundle_source_ref" ]; then
  bundle_source_ref=$(git bundle list-heads "$bundle_path" | awk '$2 == "HEAD" { print $2 }')
fi
if [ "$(printf '%s\n' "$bundle_source_ref" | sed '/^$/d' | wc -l | tr -d ' ')" != "1" ]; then
  echo "Expected exactly one bundle source ref, found: $bundle_source_ref" >&2
  exit 1
fi
git fetch "$bundle_path" "${bundle_source_ref}:${temp_ref}"
git update-ref "$target_ref" "$temp_ref"
git checkout 'test-assist/retry-extensions-tests-v2-9c47722bf5a10e3f'
# Ensure the working tree matches the updated branch
git reset --hard
# Remove the temporary bundle ref
git update-ref -d "$temp_ref"

# Push the branch to the target remote
git push 'origin' 'test-assist/retry-extensions-tests-v2-9c47722bf5a10e3f'

# Create the pull request
gh pr create --title '[test-improver] Add unit tests for RetryExtensions.AddRetryProvider' --base 'main' --head 'test-assist/retry-extensions-tests-v2-9c47722bf5a10e3f' --repo 'microsoft/testfx'

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • southcentralus0.in.applicationinsights.azure.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "southcentralus0.in.applicationinsights.azure.com"

See Network Configuration for more information.

🤖 Automated content by GitHub Copilot. Generated by the Test Improver workflow. · copilot · auto · 401 AIC · ⌖ 15 AIC · ⊞ 16.8K · [◷]( · )
Comment /test-assist to run again

Add this agentic workflow to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/test-improver.md@main

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions