Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions src/content/docs/ci-insights/setup/github-actions.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -142,10 +142,10 @@ jobs:
running when no changes are made on the code

- **Job name**: Ensure the job name is the same as the one running tests on your pull request,
allowing CI-Insights to identify them. It can be overriden with `MERGIFY_JOB_NAME` if needed.
allowing CI-Insights to identify them. It can be overridden with `MERGIFY_TEST_JOB_NAME` if needed.

### Key Environment Variables

- `MERGIFY_TOKEN`: Application key with `ci` scope for uploading test results to CI Insights
- `RUN_COUNT`: Number of times to execute each test within a single job
- `MERGIFY_JOB_NAME`: The job name reported to CI Insights
- `MERGIFY_TEST_JOB_NAME`: The job name reported to CI Insights
4 changes: 2 additions & 2 deletions src/content/docs/ci-insights/setup/jenkins.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -162,10 +162,10 @@ pipeline {
running when no changes are made on the code

- **Job name**: Ensure the job name is the same as the one running tests on your pull request,
allowing CI-Insights to identify them. It can be overriden with `MERGIFY_JOB_NAME` if needed.
allowing CI-Insights to identify them. It can be overridden with `MERGIFY_TEST_JOB_NAME` if needed.

### Key Environment Variables

- `MERGIFY_TOKEN`: Application key with `ci` scope for uploading test results to CI Insights
- `RUN_COUNT`: Number of times to execute each test within a single job
- `MERGIFY_JOB_NAME`: The job name reported to CI Insights
- `MERGIFY_TEST_JOB_NAME`: The job name reported to CI Insights
6 changes: 3 additions & 3 deletions src/content/docs/ci-insights/test-frameworks/pytest.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ If you’re using multiple environments (e.g. `py38`, `py39`, etc.), the plugin
will work for all of them as long as the token is set correctly.

If you’re running multiple Tox environments (e.g., py38, py39, etc.), we
recommend setting the `MERGIFY_JOB_NAME` environment variable to identify each
recommend setting the `MERGIFY_TEST_JOB_NAME` environment variable to identify each
environment’s report in CI Insights:

In your GitHub Actions workflow:
Expand All @@ -130,11 +130,11 @@ In your GitHub Actions workflow:
- name: Run Tox Tests
env:
MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
MERGIFY_JOB_NAME: tox-${{ matrix.python-version }}
MERGIFY_TEST_JOB_NAME: tox-${{ matrix.python-version }}
run: tox
```

:::tip
Use `MERGIFY_JOB_NAME` to make reports clearer in CI Insights, especially when
Use `MERGIFY_TEST_JOB_NAME` to make reports clearer in CI Insights, especially when
running multiple Tox environments or using a matrix.
:::
155 changes: 77 additions & 78 deletions src/content/docs/ci-insights/test-frameworks/rspec.mdx
Original file line number Diff line number Diff line change
@@ -1,41 +1,31 @@
---
title: RSpec Integration with CI Insights
description: Report your test results from RSpec tests to CI Insights
description: Report your test results from RSpec to CI Insights
---

import rspecLogo from "../../../images/ci-insights/rspec/logo.svg"
import CommonTroubleshootingTips from "./_common-troubleshooting-tips.mdx"
import GhaMergifyCiQuarantineSetup from "./_gha_mergify_ci_quarantine_setup.mdx"
import IntegrationLogo from "../../../../components/IntegrationLogo.astro"
import MergifyCIUploadStep from "../../../../components/MergifyCIUploadStep.astro"
import CIInsightsSetupNote from "../../../../components/CIInsightsSetupNote.astro"
import MergifyCIUploadStepMatrix from "../../../../components/MergifyCIUploadStepMatrix.astro"

<IntegrationLogo src={rspecLogo} alt="RSpec logo" />

This guide shows how to generate JUnit reports from your RSpec tests and upload
them to **CI Insights** using a GitHub Actions workflow.
This guide explains how to integrate RSpec with CI Insights using the
`rspec-mergify` gem. Once installed, test results are automatically uploaded to
CI Insights without any extra workflow changes.

## Generate a JUnit Report with RSpec
## Installation

RSpec supports JUnit XML output through the built-in JUnit formatter. You can
configure RSpec to output JUnit reports using command-line options or
configuration files.
You need to install the
[`rspec-mergify`](https://rubygems.org/gems/rspec-mergify) gem to automatically
upload your test results to **CI Insights**.

### Using Command Line Options
### Gemfile

```bash
bundle exec rspec --format RspecJunitFormatter --out junit.xml
```

### Installing the JUnit Formatter

First, add the JUnit formatter gem to your Gemfile:
Add the gem to your `Gemfile`:

```ruby
# Gemfile
group :test do
gem 'rspec-junit-formatter'
gem 'rspec-mergify'
end
```

Expand All @@ -45,87 +35,96 @@ Then run:
bundle install
```

### Using RSpec Configuration
### Gem Install

Add the formatter to your `.rspec` file:
Alternatively, install it directly:

```text
--format RspecJunitFormatter
--out junit.xml
```bash
gem install rspec-mergify
```

Or configure it in your `spec_helper.rb`:

```ruby
# spec/spec_helper.rb
RSpec.configure do |config|
config.add_formatter('RspecJunitFormatter', 'junit.xml')
end
```
## Modify Your Workflow

### Using Multiple Formatters
<CIInsightsSetupNote />

You can use multiple formatters simultaneously:
Your workflow should run your tests as usual while exporting the secret
`MERGIFY_TOKEN` as an environment variable. You'll need to add the following
code to the GitHub Actions step running your tests:

```bash
bundle exec rspec --format documentation --format RspecJunitFormatter --out junit.xml
```yaml
env:
MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
```

### Using Rake Task
For example:

You can also create a Rake task in your `Rakefile`:

```ruby
require 'rspec/core/rake_task'

RSpec::Core::RakeTask.new(:spec_junit) do |t|
t.rspec_opts = '--format RspecJunitFormatter --out junit.xml'
end
```yaml
- name: Run RSpec Tests 🧪
env:
MERGIFY_TOKEN: ${{ secrets.MERGIFY_TOKEN }}
run: bundle exec rspec
```

Then run:
The gem automatically collects your test results and sends them to CI Insights.

```bash
bundle exec rake spec_junit
```
Check the CI Insights dashboard afterward to view execution metrics, detect
flaky tests, and gather actionable feedback.

## Update Your GitHub Actions Workflow
## Environment Variables

<CIInsightsSetupNote />
| Variable | Purpose | Default |
|----------|---------|---------|
| `MERGIFY_TOKEN` | API authentication token | **Required** |
| `MERGIFY_API_URL` | API endpoint location | `https://api.mergify.com` |
| `RSPEC_MERGIFY_ENABLE` | Force-enable outside CI | `false` |
| `RSPEC_MERGIFY_DEBUG` | Print spans to console | `false` |
| `MERGIFY_TRACEPARENT` | W3C distributed trace context | Optional |
| `MERGIFY_TEST_JOB_NAME` | Test job name identifier | Optional |

After generating the JUnit report, add a step to upload the results to CI
Insights using the mergifyio/gha-mergify-ci action.
:::tip
The gem auto-activates in CI environments (detected via the `CI` environment
variable). To enable it outside CI, set `RSPEC_MERGIFY_ENABLE=true`.
:::

For example, in your workflow file:
:::tip
Use `MERGIFY_TEST_JOB_NAME` to make reports clearer in CI Insights, especially
when running multiple test suites or using a matrix strategy.
:::

```yaml
- name: Run RSpec Tests and Generate JUnit Report
continue-on-error: true
run: bundle exec rspec --format RspecJunitFormatter --out junit.xml
```
## Alternative: Manual JUnit XML Upload

<MergifyCIUploadStep reportPath="junit.xml" />
<MergifyCIUploadStepMatrix reportPath="junit.xml" />
If you prefer not to use the `rspec-mergify` gem, you can manually generate
JUnit XML reports and upload them using the Mergify CI action.

<GhaMergifyCiQuarantineSetup />
### Generate a JUnit Report

## Verify and Review in CI Insights
Install the JUnit formatter gem:

After pushing these changes:
```ruby
# Gemfile
group :test do
gem 'rspec-junit-formatter'
end
```

1. Your GitHub Actions workflow will execute your RSpec tests.
2. A JUnit report (junit.xml) is generated.
3. The Mergify CI action uploads the report to CI Insights.
Configure RSpec to output JUnit reports in your `.rspec` file:

You can then review your test results, including any failures or flaky tests,
directly in the [CI Insights
dashboard](https://dashboard.mergify.com/ci-insights/jobs).
```text
--format RspecJunitFormatter
--out junit.xml
```

## Troubleshooting Tips
### Upload with GitHub Actions

<ul>
<li>Gem Installation: Ensure the `rspec-junit-formatter` gem is properly installed and included in your Gemfile.</li>
<li>RSpec Configuration: Verify that the formatter is correctly configured in your RSpec setup.</li>
```yaml
- name: Run RSpec Tests
continue-on-error: true
run: bundle exec rspec --format RspecJunitFormatter --out junit.xml

<CommonTroubleshootingTips />
</ul>
- name: Upload test results to Mergify
if: success() || failure()
uses: mergifyio/gha-mergify-ci@v14
with:
token: ${{ secrets.MERGIFY_TOKEN }}
report_path: "junit.xml"
```