[573] add automated snapshot testing with Playwright and PR comment reporting#93
[573] add automated snapshot testing with Playwright and PR comment reporting#93IhorMasechko wants to merge 4 commits intomainfrom
Conversation
📝 WalkthroughWalkthroughA new GitHub Actions workflow is introduced to automate Playwright snapshot testing on pull requests. The Playwright screenshot test suite is refactored to run parameterized snapshot tests across multiple routes, with each route generating its own test and snapshot artifact, improving test coverage and artifact reporting. Changes
Sequence Diagram(s)sequenceDiagram
participant Developer
participant GitHub
participant GitHub Actions Runner
participant Docker/Playwright
participant PR
Developer->>GitHub: Opens/updates PR
GitHub->>GitHub Actions Runner: Triggers workflow
GitHub Actions Runner->>GitHub: Checks out code
GitHub Actions Runner->>Docker/Playwright: Runs snapshot tests (multiple routes)
Docker/Playwright->>GitHub Actions Runner: Generates snapshots
GitHub Actions Runner->>GitHub: Uploads snapshot artifacts
GitHub Actions Runner->>PR: Posts/updates comment with snapshot report and artifact link
GitHub Actions Runner->>Docker/Playwright: Cleans up Docker resources
Possibly related PRs
Suggested reviewers
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (1)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
🔍 Vulnerabilities of
|
| digest | sha256:a64e62f69ce07eebeeed929bbd364e50927ec02d14a92ded0dd5aedd0beb2342 |
| vulnerabilities |     |
| platform | linux/amd64 |
| size | 283 MB |
| packages | 934 |
📦 Base Image node:23-alpine
| also known as |
|
| digest | sha256:169ee1b69bd3f5c7c8508a9919e8b367bba9dc7fa9a03097ec9f85d2e8179631 |
| vulnerabilities |  |
Description
| ||||||||||||
Description
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
website/e2e/tests/screenshot-test.spec.js (2)
9-37: Good test implementation with a few enhancement opportunities.The test implementation properly handles waiting for page load events and font rendering before taking snapshots. The parameterized approach efficiently creates tests for each route.
Consider these enhancements:
- Add responsive testing with multiple viewport sizes:
- await page.setViewportSize({ width: 1280, height: 720 }); + // Test multiple viewport sizes for responsive design + const viewports = [ + { width: 1280, height: 720 }, // Desktop + { width: 768, height: 1024 }, // Tablet + { width: 375, height: 812 } // Mobile + ]; + for (const viewport of viewports) { + await page.setViewportSize(viewport); + // Take and verify snapshot for each viewport size + const snapshot = await page.screenshot({ + fullPage: true, + timeout: 30000, + }); + expect(snapshot).toMatchSnapshot(`${viewport.width}x${viewport.height}-${name}`, { + maxDiffPixels: 300, + threshold: 0.5, + }); + }
- Document why the specific tolerance values were chosen:
expect(snapshot).toMatchSnapshot(name, { maxDiffPixels: 300, // Allow for minor differences that don't affect functionality threshold: 0.5, // 0.5 tolerance level for color differences });
32-34: Consider adding documentation for snapshot comparison tolerances.The
maxDiffPixelsandthresholdvalues control how strict the snapshot comparison is. It would be valuable to document why these specific values were chosen to help future maintainers understand the rationale.expect(snapshot).toMatchSnapshot(name, { - maxDiffPixels: 300, - threshold: 0.5, + maxDiffPixels: 300, // Allows for small differences in rendering across environments + threshold: 0.5, // Tolerance for pixel color intensity differences (0-1 scale) });.github/workflows/playwright-reports.yml (1)
39-42: Ensure directory paths are consistent.There's an inconsistency between the paths used in the "Ensure snapshot dirs are writable" step and this step, which could lead to confusion.
- mkdir -p website/e2e/pr-snapshots/screenshot-test.spec.js-snapshots + # Create consistent directory path - align with earlier step or document why they differ + mkdir -p website/e2e/pr-snapshots/screenshot-test.spec.js-snapshots + echo "Note: Snapshot directory paths differ between steps because we're copying from container path to a PR-specific local path"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/playwright-reports.yml(1 hunks)website/e2e/tests/screenshot-test.spec.js(1 hunks)
🔇 Additional comments (5)
website/e2e/tests/screenshot-test.spec.js (1)
3-7: Well-structured route parameterization.The array of route objects is a clean way to define test paths and corresponding snapshot names. This approach makes it easy to add more routes in the future.
.github/workflows/playwright-reports.yml (4)
3-16: Well-configured workflow triggers and permissions.The workflow is correctly set up to run on appropriate PR events with proper permissions. Good job excluding dependabot runs to avoid unnecessary test executions.
52-66: Good PR comment formatting and artifact linking.The comment structure with emojis and markdown formatting is very readable and provides clear access to artifacts.
68-84: Robust PR comment update mechanism.The approach for finding and updating an existing comment is well-implemented. It effectively prevents duplicate comments on the PR while keeping information current.
86-89: Thorough Docker cleanup.The cleanup step properly removes Docker resources, which is important for maintaining a clean CI environment and preventing resource leaks.
|
|
Closing this PR for now — I’ll come back to it and continue the work at a later time. |
1 participant
Summary
This PR introduces automated E2E screenshot (snapshot) testing using Playwright. Snapshots are taken on key pages and compared against baseline images.
Key Features
How It Works
Notes
e2e/tests/screenshot-test.spec.js-snapshotsgh)chownto UID 1000Next Steps / Future Enhancements