diff --git a/src/content/docs/ci-insights/setup/github-actions.mdx b/src/content/docs/ci-insights/setup/github-actions.mdx index b5d93e69d7..bb1868babb 100644 --- a/src/content/docs/ci-insights/setup/github-actions.mdx +++ b/src/content/docs/ci-insights/setup/github-actions.mdx @@ -142,10 +142,10 @@ jobs: running when no changes are made on the code - **Job name**: Ensure the job name is the same as the one running tests on your pull request, - allowing CI-Insights to identify them. It can be overriden with `MERGIFY_JOB_NAME` if needed. + allowing CI-Insights to identify them. It can be overridden with `MERGIFY_TEST_JOB_NAME` if needed. ### Key Environment Variables - `MERGIFY_TOKEN`: Application key with `ci` scope for uploading test results to CI Insights - `RUN_COUNT`: Number of times to execute each test within a single job -- `MERGIFY_JOB_NAME`: The job name reported to CI Insights +- `MERGIFY_TEST_JOB_NAME`: The job name reported to CI Insights diff --git a/src/content/docs/ci-insights/setup/jenkins.mdx b/src/content/docs/ci-insights/setup/jenkins.mdx index e8aa836df7..a72fcba151 100644 --- a/src/content/docs/ci-insights/setup/jenkins.mdx +++ b/src/content/docs/ci-insights/setup/jenkins.mdx @@ -162,10 +162,10 @@ pipeline { running when no changes are made on the code - **Job name**: Ensure the job name is the same as the one running tests on your pull request, - allowing CI-Insights to identify them. It can be overriden with `MERGIFY_JOB_NAME` if needed. + allowing CI-Insights to identify them. It can be overridden with `MERGIFY_TEST_JOB_NAME` if needed. ### Key Environment Variables - `MERGIFY_TOKEN`: Application key with `ci` scope for uploading test results to CI Insights - `RUN_COUNT`: Number of times to execute each test within a single job -- `MERGIFY_JOB_NAME`: The job name reported to CI Insights +- `MERGIFY_TEST_JOB_NAME`: The job name reported to CI Insights diff --git a/src/content/docs/ci-insights/test-frameworks/pytest.mdx b/src/content/docs/ci-insights/test-frameworks/pytest.mdx index 6153c8f1e7..56c65afda9 100644 --- a/src/content/docs/ci-insights/test-frameworks/pytest.mdx +++ b/src/content/docs/ci-insights/test-frameworks/pytest.mdx @@ -121,7 +121,7 @@ If you’re using multiple environments (e.g. `py38`, `py39`, etc.), the plugin will work for all of them as long as the token is set correctly. If you’re running multiple Tox environments (e.g., py38, py39, etc.), we -recommend setting the `MERGIFY_JOB_NAME` environment variable to identify each +recommend setting the `MERGIFY_TEST_JOB_NAME` environment variable to identify each environment’s report in CI Insights: In your GitHub Actions workflow: @@ -130,11 +130,11 @@ In your GitHub Actions workflow: - name: Run Tox Tests env: MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} - MERGIFY_JOB_NAME: tox-${{ matrix.python-version }} + MERGIFY_TEST_JOB_NAME: tox-${{ matrix.python-version }} run: tox ``` :::tip -Use `MERGIFY_JOB_NAME` to make reports clearer in CI Insights, especially when +Use `MERGIFY_TEST_JOB_NAME` to make reports clearer in CI Insights, especially when running multiple Tox environments or using a matrix. ::: diff --git a/src/content/docs/ci-insights/test-frameworks/rspec.mdx b/src/content/docs/ci-insights/test-frameworks/rspec.mdx index 09f3075b82..6690038f42 100644 --- a/src/content/docs/ci-insights/test-frameworks/rspec.mdx +++ b/src/content/docs/ci-insights/test-frameworks/rspec.mdx @@ -1,41 +1,31 @@ --- title: RSpec Integration with CI Insights -description: Report your test results from RSpec tests to CI Insights +description: Report your test results from RSpec to CI Insights --- import rspecLogo from "../../../images/ci-insights/rspec/logo.svg" -import CommonTroubleshootingTips from "./_common-troubleshooting-tips.mdx" -import GhaMergifyCiQuarantineSetup from "./_gha_mergify_ci_quarantine_setup.mdx" import IntegrationLogo from "../../../../components/IntegrationLogo.astro" -import MergifyCIUploadStep from "../../../../components/MergifyCIUploadStep.astro" import CIInsightsSetupNote from "../../../../components/CIInsightsSetupNote.astro" -import MergifyCIUploadStepMatrix from "../../../../components/MergifyCIUploadStepMatrix.astro" -This guide shows how to generate JUnit reports from your RSpec tests and upload -them to **CI Insights** using a GitHub Actions workflow. +This guide explains how to integrate RSpec with CI Insights using the +`rspec-mergify` gem. Once installed, test results are automatically uploaded to +CI Insights without any extra workflow changes. -## Generate a JUnit Report with RSpec +## Installation -RSpec supports JUnit XML output through the built-in JUnit formatter. You can -configure RSpec to output JUnit reports using command-line options or -configuration files. +You need to install the +[`rspec-mergify`](https://rubygems.org/gems/rspec-mergify) gem to automatically +upload your test results to **CI Insights**. -### Using Command Line Options +### Gemfile -```bash -bundle exec rspec --format RspecJunitFormatter --out junit.xml -``` - -### Installing the JUnit Formatter - -First, add the JUnit formatter gem to your Gemfile: +Add the gem to your `Gemfile`: ```ruby -# Gemfile group :test do - gem 'rspec-junit-formatter' + gem 'rspec-mergify' end ``` @@ -45,87 +35,96 @@ Then run: bundle install ``` -### Using RSpec Configuration +### Gem Install -Add the formatter to your `.rspec` file: +Alternatively, install it directly: -```text ---format RspecJunitFormatter ---out junit.xml +```bash +gem install rspec-mergify ``` -Or configure it in your `spec_helper.rb`: - -```ruby -# spec/spec_helper.rb -RSpec.configure do |config| - config.add_formatter('RspecJunitFormatter', 'junit.xml') -end -``` +## Modify Your Workflow -### Using Multiple Formatters + -You can use multiple formatters simultaneously: +Your workflow should run your tests as usual while exporting the secret +`MERGIFY_TOKEN` as an environment variable. You'll need to add the following +code to the GitHub Actions step running your tests: -```bash -bundle exec rspec --format documentation --format RspecJunitFormatter --out junit.xml +```yaml +env: + MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} ``` -### Using Rake Task +For example: -You can also create a Rake task in your `Rakefile`: - -```ruby -require 'rspec/core/rake_task' - -RSpec::Core::RakeTask.new(:spec_junit) do |t| - t.rspec_opts = '--format RspecJunitFormatter --out junit.xml' -end +```yaml +- name: Run RSpec Tests 🧪 + env: + MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }} + run: bundle exec rspec ``` -Then run: +The gem automatically collects your test results and sends them to CI Insights. -```bash -bundle exec rake spec_junit -``` +Check the CI Insights dashboard afterward to view execution metrics, detect +flaky tests, and gather actionable feedback. -## Update Your GitHub Actions Workflow +## Environment Variables - +| Variable | Purpose | Default | +|----------|---------|---------| +| `MERGIFY_TOKEN` | API authentication token | **Required** | +| `MERGIFY_API_URL` | API endpoint location | `https://api.mergify.com` | +| `RSPEC_MERGIFY_ENABLE` | Force-enable outside CI | `false` | +| `RSPEC_MERGIFY_DEBUG` | Print spans to console | `false` | +| `MERGIFY_TRACEPARENT` | W3C distributed trace context | Optional | +| `MERGIFY_TEST_JOB_NAME` | Test job name identifier | Optional | -After generating the JUnit report, add a step to upload the results to CI -Insights using the mergifyio/gha-mergify-ci action. +:::tip +The gem auto-activates in CI environments (detected via the `CI` environment +variable). To enable it outside CI, set `RSPEC_MERGIFY_ENABLE=true`. +::: -For example, in your workflow file: +:::tip +Use `MERGIFY_TEST_JOB_NAME` to make reports clearer in CI Insights, especially +when running multiple test suites or using a matrix strategy. +::: -```yaml -- name: Run RSpec Tests and Generate JUnit Report - continue-on-error: true - run: bundle exec rspec --format RspecJunitFormatter --out junit.xml -``` +## Alternative: Manual JUnit XML Upload - - +If you prefer not to use the `rspec-mergify` gem, you can manually generate +JUnit XML reports and upload them using the Mergify CI action. - +### Generate a JUnit Report -## Verify and Review in CI Insights +Install the JUnit formatter gem: -After pushing these changes: +```ruby +# Gemfile +group :test do + gem 'rspec-junit-formatter' +end +``` -1. Your GitHub Actions workflow will execute your RSpec tests. -2. A JUnit report (junit.xml) is generated. -3. The Mergify CI action uploads the report to CI Insights. +Configure RSpec to output JUnit reports in your `.rspec` file: -You can then review your test results, including any failures or flaky tests, -directly in the [CI Insights -dashboard](https://dashboard.mergify.com/ci-insights/jobs). +```text +--format RspecJunitFormatter +--out junit.xml +``` -## Troubleshooting Tips +### Upload with GitHub Actions -
    -
  • Gem Installation: Ensure the `rspec-junit-formatter` gem is properly installed and included in your Gemfile.
  • -
  • RSpec Configuration: Verify that the formatter is correctly configured in your RSpec setup.
  • +```yaml +- name: Run RSpec Tests + continue-on-error: true + run: bundle exec rspec --format RspecJunitFormatter --out junit.xml - -
+- name: Upload test results to Mergify + if: success() || failure() + uses: mergifyio/gha-mergify-ci@v14 + with: + token: ${{ secrets.MERGIFY_TOKEN }} + report_path: "junit.xml" +```