diff --git a/src/components/MergifyCIUploadStep.astro b/src/components/MergifyCIUploadStep.astro index c377bdc605..688525b1da 100644 --- a/src/components/MergifyCIUploadStep.astro +++ b/src/components/MergifyCIUploadStep.astro @@ -13,7 +13,8 @@ const { reportPath } = Astro.props; uses: mergifyio/gha-mergify-ci@v8 with: token: \${{ secrets.MERGIFY_TOKEN }} - report_path: ${reportPath}`} + report_path: ${reportPath} + test_step_outcome: \${{ steps.tests.outcome }}`} lang="yaml" theme="slack-ochin" /> @@ -27,4 +28,11 @@ const { reportPath } = Astro.props; : Points to where your JUnit file is located. Make sure it matches the path you set in your CI job. +
  • + : + Passes the test runner step's outcome so Mergify can detect silent failures where the runner + crashed but the JUnit report appears clean. Add an (such + as ) to your test runner step and + update the reference to match. +
  • diff --git a/src/components/MergifyCIUploadStepMatrix.astro b/src/components/MergifyCIUploadStepMatrix.astro index da8d7136df..a78f13cacb 100644 --- a/src/components/MergifyCIUploadStepMatrix.astro +++ b/src/components/MergifyCIUploadStepMatrix.astro @@ -33,7 +33,8 @@ const { reportPath } = Astro.props; with: job_name: example_matrix (\${{ matrix.version }}) token: \${{ secrets.MERGIFY_TOKEN }} - report_path: ${reportPath}`} + report_path: ${reportPath} + test_step_outcome: \${{ steps.tests.outcome }}`} lang="yaml" theme="slack-ochin" /> diff --git a/src/content/docs/ci-insights/test-frameworks/_gha_mergify_ci_quarantine_setup.mdx b/src/content/docs/ci-insights/test-frameworks/_gha_mergify_ci_quarantine_setup.mdx index 2c00119e84..3962220913 100644 --- a/src/content/docs/ci-insights/test-frameworks/_gha_mergify_ci_quarantine_setup.mdx +++ b/src/content/docs/ci-insights/test-frameworks/_gha_mergify_ci_quarantine_setup.mdx @@ -1,4 +1,10 @@ In order to benefit from CI Insights Quarantine, you need to add `continue-on-error: true` -in your GitHb Actions step that execute your tests and generates the JUnit file. +in your GitHub Actions step that executes your tests and generates the JUnit file. The step running the `gha-mergify-ci` action will determine the success or failure conclusion, considering quarantined tests. + +You should also pass `test_step_outcome: ${{ steps.tests.outcome }}` to the step that runs +`mergifyio/gha-mergify-ci` (where `tests` is the `id` of your test runner step) to detect +silent failures where the test runner crashed but the JUnit report appears clean. Without +this input, a crash that produces a partial or empty report could be mistakenly treated as a +success.