d2l-test-reporting

Helper package for generating reports for use with the D2L test reporting framework. This package is meant to be used in conjunction with this GitHub Action.

Note

If you have any questions, concerns or just want to chat feel free to reach out in #test-reporting (D2L employee accessible only).

Installation

npm install d2l-test-reporting

Usage

This library provides reporters for many of the test execution frameworks we use at D2L. In order to use the accompanying GitHub Action to upload reports to the back-end you must use of one of the reporters provided below or have a custom implementation that outputs a JSON file in the format outlined in Report Format.

Reporters

Mocha

Please consult the official documentation for Mocha to see how to use reporters. Below is an example of how to add the reporter provided by this package. It assumes you are using the default .mocharc.js file for configuration.

module.exports = {
  spec: 'test/*.test.js',
  reporter: 'd2l-test-reporting/reporters/mocha.js',
  reporterOptions: [
    'reportPath=./d2l-test-report.json', // optional
    'reportConfigurationPath=./d2l-test-reporting.config.json' // optional
  ]
};

Inputs

• reportPath: path to output the report to, relative to current working directory. Not required. Defaults to ./d2l-test-report.json.
• reportConfigurationPath: path to the D2L test reporting configuration file for mapping test type, experience and tool to test code. Not required. Defaults to ./d2l-test-reporting.config.json.

Playwright

Please consult the official documentation for Playwright to see how to use reporters. Below is an example of how to add the reporter provided by this package. It assumes you are using the default playwright.config.js file for configuration.

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  reporter: [
    [
      'd2l-test-reporting/reporters/playwright.js',
      {
        reportPath: './d2l-test-report.json', // optional
        reportConfigurationPath: './d2l-test-reporting.config.json' // optional
      }
    ],
    ['list']
  ],
  testDir: '../',
  testMatch: '*.test.js',
  projects: [{
    name: 'firefox',
    use: devices['Desktop Firefox'],
    testMatch: 'firefox/*.test.js'
  }]
});

Warning

Currently the merge-reports command is not fully supported due to a lack of browser/launcher information preservation with the blob reporter. If you are using a GitHub matrix run this may result in partial data showing in the reporting dashboard as it becomes available.

Inputs

• reportPath: path to output the report to, relative to current working directory. Not required. Defaults to ./d2l-test-report.json.
• reportConfigurationPath: path to the D2L test reporting configuration file for mapping test type, experience and tool to test code. Not required. Defaults to ./d2l-test-reporting.config.json.

@web/test-runner

Please consult the official documentation for @web/test-runner to see how to use reporters. Below is an example of how to add the reporter provided by this package. It assumes you are using the default web-test-runner.config.js file for configuration.

import { defaultReporter } from '@web/test-runner';
import { reporter } from 'd2l-test-reporting/reporters/web-test-runner.js';

export default {
  reporters: [
    defaultReporter(),
    reporter({
      reportPath: './d2l-test-report.json', // optional
      reportConfigurationPath: './d2l-test-reporting.config.json' // optional
    })
  ],
  files: 'test/component-*.test.js',
  groups: [{
    name: 'group',
    files: 'test/group/component-*.test.js'
  }]
};

Inputs

• reportPath: path to output the report to, relative to current working directory. Not required. Defaults to ./d2l-test-report.json.
• reportConfigurationPath: path to the D2L test reporting configuration file for mapping test type, experience and tool to test code. Not required. Defaults to ./d2l-test-reporting.config.json.

Custom Reporters

If your test framework isn't supported by the built-in reporters, you can build your own using the ReportBuilder class.

See the ReportBuilder Guide for a complete walkthrough of creating a custom reporter for any test framework.

Configuration

To have the test type, experience and tool mapped to test code, a D2L test reporting configuration file is required when using one of the reporters provided in this package.

Below are examples of how to create the config file. Note that the type field will end up as lowercase in the report.

Please see Automated Testing Definitions on Confluence for the list of test types that should be used when creating the D2L test reporting configuration file.

{
  "type": "UI Visual Diff",
  "experience": "Experience",
  "tool": "Tool",
}

{
  "type": "UI E2E",
  "overrides": [
    {
      "pattern": "tests/account-settings/**/*",
      "experience": "Administration",
      "tool": "Account Settings"
    },
    {
      "pattern": "tests/announcements/**/*",
      "experience": "Teaching & Learning",
      "tool": "Announcements"
    },
    {
      "pattern": "tests/rubrics.test.js",
      "experience": "Assessment",
      "tool": "Rubrics"
    }
  ]
}

Report Format

For details on the schema that the various custom reporters output, see Report Format.

Note

This specific JSON structure is required in order to upload reports to the back-end using the corresponding GitHub Action.

Developing

After cloning the repository, make sure to install dependencies.

npm ci
npx playwright install --with-deps
npx puppeteer browsers install chrome

Linting

# eslint and editorconfig-checker
npm run lint

# eslint only
npm run lint:eslint

# editorconfig-checker only
npm run lint:editorconfig

Fixing

# currently only eslint
npm run fix

# eslint only
npm run fix:eslint

Testing

# lint, unit tests and integration tests
npm test

# unit tests and integration tests only
npm run test:all

# unit tests only
npm run test:unit

# integration tests only
npm run test:integration

Versioning and Releasing

This repo is configured to use semantic-release. Commits prefixed with fix: and feat: will trigger patch and minor releases when merged to main.

To learn how to create major releases and release from maintenance branches, refer to the semantic-release GitHub Action documentation.