Skip to content

Make playwright test environment more dynamic#2532

Merged
jawelton74 merged 24 commits into
masterfrom
users/jawelton/e2e-test-setup-062626
Jul 20, 2026
Merged

Make playwright test environment more dynamic#2532
jawelton74 merged 24 commits into
masterfrom
users/jawelton/e2e-test-setup-062626

Conversation

@jawelton74

@jawelton74 jawelton74 commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Preview this branch

This change introduces changes to the pipelines and test setup to make the environment configuration more dynamic and support easier setup of new test environments as needed.

  • All of the test resources are configured under Actions secrets with E2ETESTS_ prefixes.
  • Make the NoSQL test account token retrieval smarter so that it only grabs the one it needs for that shard.
  • Preview site and the storage account that host PR builds and test results are now dynamic.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
20 14 0 155s

📁 Report: 28890344999-3/report.zip
Open container (Azure sign-in required) → click into 28890344999-3 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 20 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
80 555 0 318s

📁 Report: 28892308623-1/report.zip
Open container (Azure sign-in required) → click into 28892308623-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 20 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
80 555 0 4187s

📁 Report: 28892725881-1/report.zip
Open container (Azure sign-in required) → click into 28892725881-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 20 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
62 450 0 4215s

📁 Report: 28897861115-1/report.zip
Open container (Azure sign-in required) → click into 28897861115-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 20 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

Comment thread test/fx.ts Fixed
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
80 16 0 2599s

📁 Report: 28959869942-1/report.zip
Open container (Azure sign-in required) → click into 28959869942-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 6 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
80 504 0 6130s

📁 Report: 28958185106-1/report.zip
Open container (Azure sign-in required) → click into 28958185106-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 20 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
80 16 0 2643s

📁 Report: 28964038437-1/report.zip
Open container (Azure sign-in required) → click into 28964038437-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 6 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
0 16 0 2584s

📁 Report: 28974524758-1/report.zip
Open container (Azure sign-in required) → click into 28974524758-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 16 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
0 1 0 2490s

📁 Report: 28974524758-2/report.zip
Open container (Azure sign-in required) → click into 28974524758-2 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 16 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
4 12 0 172s

📁 Report: 28974524758-3/report.zip
Open container (Azure sign-in required) → click into 28974524758-3 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 12 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
0 1 0 419s

📁 Report: 28974524758-4/report.zip
Open container (Azure sign-in required) → click into 28974524758-4 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 12 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
1 0 0 97s

📁 Report: 28974524758-5/report.zip
Open container (Azure sign-in required) → click into 28974524758-5 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 11 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
0 1 0 379s

📁 Report: 28974524758-6/report.zip
Open container (Azure sign-in required) → click into 28974524758-6 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 11 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
1 0 0 80s

📁 Report: 28974524758-7/report.zip
Open container (Azure sign-in required) → click into 28974524758-7 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 10 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
1 0 0 74s

📁 Report: 28974524758-8/report.zip
Open container (Azure sign-in required) → click into 28974524758-8 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 9 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
0 0 1 104s

📁 Report: 28974524758-9/report.zip
Open container (Azure sign-in required) → click into 28974524758-9 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 8 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
1 0 0 94s

📁 Report: 28974524758-10/report.zip
Open container (Azure sign-in required) → click into 28974524758-10 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 7 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
1 0 0 90s

📁 Report: 28974524758-11/report.zip
Open container (Azure sign-in required) → click into 28974524758-11 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 6 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
101 39 0 3916s

📁 Report: 29123530283-1/report.zip
Open container (Azure sign-in required) → click into 29123530283-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 8 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
599 62 5 6749s

📁 Report: 29124090132-1/report.zip
Open container (Azure sign-in required) → click into 29124090132-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 12 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
635 28 4 3891s

📁 Report: 29124090132-2/report.zip
Open container (Azure sign-in required) → click into 29124090132-2 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 12 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
640 14 9 2676s

📁 Report: 29284654102-1/report.zip
Open container (Azure sign-in required) → click into 29284654102-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 7 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

Copy link
Copy Markdown

Playwright tests ✅ passed

Passed Failed Flaky Duration
220 0 9 558s

📁 Report: 29284654102-2/report.zip
Open container (Azure sign-in required) → click into 29284654102-2 folder → click report.zipDownload → unzip → open index.html · Workflow run

@github-actions

Copy link
Copy Markdown

Playwright tests ✅ passed

Passed Failed Flaky Duration
647 0 16 1018s

📁 Report: 29294105611-1/report.zip
Open container (Azure sign-in required) → click into 29294105611-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

@github-actions

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
648 1 14 918s

📁 Report: 29294135548-1/report.zip
Open container (Azure sign-in required) → click into 29294135548-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 1 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

Copy link
Copy Markdown

Playwright tests ✅ passed

Passed Failed Flaky Duration
647 0 16 963s

📁 Report: 29294135548-3/report.zip
Open container (Azure sign-in required) → click into 29294135548-3 folder → click report.zipDownload → unzip → open index.html · Workflow run

@github-actions

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
651 1 11 919s

📁 Report: 29602814042-1/report.zip
Open container (Azure sign-in required) → click into 29602814042-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 1 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@jawelton74 jawelton74 changed the title Update Az CLI to use new secrets. Make playwright test environment more dynamic Jul 17, 2026
@github-actions

Copy link
Copy Markdown

Playwright tests ✅ passed

Passed Failed Flaky Duration
33 0 1 319s

📁 Report: 29602814042-2/report.zip
Open container (Azure sign-in required) → click into 29602814042-2 folder → click report.zipDownload → unzip → open index.html · Workflow run

@jawelton74
jawelton74 marked this pull request as ready for review July 17, 2026 18:42
@jawelton74
jawelton74 requested a review from a team as a code owner July 17, 2026 18:42
@github-actions

Copy link
Copy Markdown

Playwright tests ❌ failed

Passed Failed Flaky Duration
652 1 10 874s

📁 Report: 29604564021-1/report.zip
Open container (Azure sign-in required) → click into 29604564021-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

⚠️ 1 shard(s) failed before tests ran (infra/auth issue). Stats below reflect only shards that executed.

@github-actions

Copy link
Copy Markdown

Playwright tests ✅ passed

Passed Failed Flaky Duration
651 0 12 946s

📁 Report: 29605219863-1/report.zip
Open container (Azure sign-in required) → click into 29605219863-1 folder → click report.zipDownload → unzip → open index.html · Workflow run

@jawelton74
jawelton74 merged commit e3b77ec into master Jul 20, 2026
33 checks passed
@jawelton74
jawelton74 deleted the users/jawelton/e2e-test-setup-062626 branch July 20, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants