-
Notifications
You must be signed in to change notification settings - Fork 593
Fix codecov configuration to properly enforce coverage thresholds #5323
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Co-authored-by: guhetier <[email protected]>
Co-authored-by: guhetier <[email protected]>
Co-authored-by: guhetier <[email protected]>
codecov.yml
…ient Co-authored-by: guhetier <[email protected]>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #5323 +/- ##
==========================================
- Coverage 86.58% 86.40% -0.19%
==========================================
Files 59 59
Lines 18331 18331
==========================================
- Hits 15872 15838 -34
- Misses 2459 2493 +34 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Note to reviewer: we may have to monitor / confirm this is behaving as we want on PR with more churn. |
The codecov configuration was causing CI failures even when coverage changes were within the configured 2% threshold. This affected PRs like #5315 where project coverage moved by less than 2% but CI still failed.
Problem
The issue had two root causes:
patch.threshold: 2%
setting allowed patch coverage to drop by 2%, but the requirement is that patches should maintain at least 80% coveragefail_ci_if_error: true
setting in codecov-action was causing premature CI failures, potentially overriding codecov's own threshold logicSolution
Fixed codecov.yml configuration
Updated GitHub workflow
Changed
fail_ci_if_error: false
in the codecov-action to rely on codecov.yml thresholds instead of failing on upload errors that might mask legitimate threshold-based passes.Expected Behavior
After this fix:
This ensures that codecov properly handles the randomized test behavior mentioned in the issue while still enforcing the required coverage standards.
Fixes #5322.
💬 Share your feedback on Copilot coding agent for the chance to win a $200 gift card! Click here to start the survey.