v0.44.0

k6 v0.44.0 is here! 🎉 This release includes:

• A new k6/experimental/webcrypto module implementing (partially) the Web Crypto API specification.
• A sampling option for the experimental tracing module.
• Memory usage improvements.
• Bug fixes and UX improvements.

Some highlights from the k6/experimental/browser module are:

• locator.click is now asynchronous, which is a breaking change.
• browserContext.addCookies has now been implemented.
• browserType.Connect has been implemented so k6 can now connect to an already running Chrome/Chromium browser instance.
• Web vitals are natively supported when working with the browser module.

Breaking changes

The browser module is still in an experimental stage, and therefore breaking changes are expected as we are improving the APIs to make them more user-friendly.

• browser#790 Converted locator.click to async to have feature parity with page.click and elementHandle.click. Users must remember to work with promise.All and page.waitForNavigation() when a click action results in navigation.

  A locator.click action that doesn't result in navigation can be used like so:

  const tails = page.locator("input[value='Bet on tails!']");
  await tails.click(),

  A locator.click action that does result in a navigation can be used like so:

  const tails = page.locator("input[value='Bet on tails!']");
  await Promise.all([
    page.waitForNavigation(),
    tails.click(),
  ]);

• browser#817 We've removed --no-sandbox from the default Chrome launch arguments. Now Chrome will launch with a sandbox, which is a more secure way of running the browser. If you are running tests under a root user, the browser will no longer launch unless the --no-sandbox argument is supplied. You can still pass this flag when launching a new Chrome instance using the args parameter on chromium.launch:

  const browser = chromium.launch({
    args: ['no-sandbox'],
  });

• browser#844 Removed the exported version param from the root module. Users should from now on reference the k6 version instead of the browser module version.

• browser#838 Removed the first meaningful paint metric. This metric is being deprecated across all the browsers, because the metric's definition relies on browser-specific implementation details, and we've now introduced web vitals in the browser module which is a reliable industry standard way to measure frontend performance. You can find more details here.

• browser#843 Removed the build step from Github Actions. From this release onwards, no new standalone browser binaries will be built and available from the releases section. The latest version of the browser module will be available in the k6 binary which can be found in the k6 releases page.

New features

A new k6/experimental/webcrypto module implementing the Web Crypto API specification #3007

This release includes a new k6/experimental/webcrypto module partially implementing the Web Crypto API specification in k6.

Expand to see an example of the new functionality.

This example shows encrypting and decrypting of a "Hello, World!" string using AES-CBC algorithm.

import { crypto } from 'k6/experimental/webcrypto';

export default async function () {
  const key = await crypto.subtle.generateKey(
    {
      name: 'AES-CBC',
      length: 256,
    },
    true,
    ['encrypt', 'decrypt']
  );

  const encoded = stringToArrayBuffer('Hello, World!');
  const iv = crypto.getRandomValues(new Uint8Array(16));

  const ciphertext = await crypto.subtle.encrypt(
    {
      name: 'AES-CBC',
      iv: iv,
    },
    key,
    encoded
  );

  const plaintext = await crypto.subtle.decrypt(
    {
      name: 'AES-CBC',
      iv: iv,
    },
    key,
    ciphertext
  );

  console.log(
    'deciphered text == original text: ',
    arrayBufferToHex(plaintext) === arrayBufferToHex(encoded)
  );
}

function arrayBufferToHex(buffer) {
  return [...new Uint8Array(buffer)].map((x) => x.toString(16).padStart(2, '0')).join('');
}

function stringToArrayBuffer(str) {
  var buf = new ArrayBuffer(str.length * 2); // 2 bytes for each char
  var bufView = new Uint16Array(buf);
  for (var i = 0, strLen = str.length; i < strLen; i++) {
    bufView[i] = str.charCodeAt(i);
  }
  return buf;
}

You can see the list of currently supported APIs and algorithms in the project's README. Documentation for the module is available here.

Add sampling capabilities to the experimental tracing module #2886

This release adds sampling capabilities to the tracing module. You can now specify a sampling rate with the sampling option when initializing a Client, or in the tracing.InstrumentHTTP function.

browserContext.addCookies browser#760

Cookies can now be added to a BrowserContext and all new Pages created from this context will have the cookie assigned to them. Thanks @zucchinho for implementing this feature!

const context = browser.newContext()
context.addCookies([{name: 'myCookie', value: 'hello world', url: 'https://test.k6.io'}])
const page = context.newPage()

browserType.Connect browser#800

There are cases where the user may want to connect to a remote browser instance where they have more control over the browser lifecycle, such as when working in a resource bound environment. This feature enables users to connect to a manually started Chrome/Chromium browser instance. It's a simple case of replacing browser.launch with browser.connect and supplying the CDP url as the first argument. Not all launch options will work with connect since the browser instance should already have started prior to working with connect. Since we assume that the user had decided to take ownership of starting the browser, we have made browser.close a NOOP when working with browser.connect, so the user will need to close the browser themselves.

const browser = chromium.connect('ws://127.0.0.1:1234/devtools/browser/e3bb7e53-ad0f-46f3-ae89-a8416868f4ce')
const page = browser.newPage();

Web Vitals are now natively supported by the browser module browser#836 browser#847

Web vitals are the defacto way for developers to measure their frontend performance using the core metrics:

• Largest contentful paint (LCP)
• First input delay (FID)
• Cumulative layout shift (CLS)

These measurements are now calculated for all tests without any additional work from your side. Simply run your test as you have been doing and you will be presented with the new metrics in the output. This is the output after running the examples/fillform.js script:

webvital_cumulative_layout_shift..........: avg=0        min=0        med=0        max=0        p(90)=0        p(95)=0
webvital_cumulative_layout_shift_good.....: 1       0.323332/s
webvital_first_contentful_paint...........: avg=278.86ms min=141.1ms  med=229.39ms max=466.1ms  p(90)=418.76ms p(95)=442.43ms
webvital_first_contentful_paint_good......: 3       0.969995/s
webvital_first_input_delay................: avg=300µs    min=200µs    med=300µs    max=399.99µs p(90)=379.99µs p(95)=389.99µs
webvital_first_input_delay_good...........: 2       0.646663/s
webvital_interaction_to_next_paint........: avg=16ms     min=16ms     med=16ms     max=16ms     p(90)=16ms     p(95)=16ms
webvital_interaction_to_next_paint_good...: 1       0.323332/s
webvital_largest_content_paint............: avg=303.6ms  min=141.1ms  med=303.6ms  max=466.1ms  p(90)=433.6ms  p(95)=449.85ms
webvital_largest_content_paint_good.......: 2       0.646663/s
webvital_time_to_first_byte...............: avg=205.23ms min=104.79ms med=188.39ms max=322.5ms  p(90)=295.67ms p(95)=309.08ms
webvital_time_to_first_byte_good..........: 3       0.969995/s

You may have noticed other metrics in there too. We rely on the web-vitals JS library which exposes a few more metrics, so we've left them in for you to experiment with. You can find more details on all the browser module metrics in our documentation.

You will no longer see browser_first_contentful_paint in the summary, and instead you can work with webvital_first_contentful_paint.

UX improvements and enhancements

• #2906 Added multiple date-time formats for CSV output. Thanks, @Azanul!
• #2916 k6 started to show the actual binary's name in the usage help. Thanks, @ariasmn!
• #2942 Reference 'k6 cloud' instead of 'Load Impact' in docs and errors.
• [#2985](https://github.com/grafana/k6...

v0.43.1

k6 v0.43.1 is a patch release containing a few bugfixes:

• #2926 fixed a panic in setup() code when vu.iterationInScenario from k6/execution was used.
• #2934 fixed a wrongly printed internal output ID to the stdout UI.
• #2938 fixed a synchronization bug that caused k6 to get stuck after the end-of-test summary when sending the usage report took more than 3s. Thanks for reporting this, @ichasepucks!

v0.43.0

k6 v0.43.0 is here! 🎉

Notable changes in this release include:

• xk6-browser is now bundled in k6 as an experimental module, and usable without a separate binary or compilation step!
• Native support for JavaScript's async/await.
• A new experimental module for distributed tracing.
• Large refactoring of core components to simplify the code base, improve maintainability, and increase test coverage.
• Bug fixes, UX improvements, and maintenance.

Keep reading for the details.

Breaking changes

• #2807 Use non-zero exit codes for tests aborted by Ctrl+C or the REST API.

  Aborting a test run with Ctrl+C will now exit with code 105, and stopping via the REST API will exit with code 103.

New Features

xk6-browser is now a built-in module #2884

This release includes xk6-browser as an experimental module. This means you can now also use the main k6 binary for browser automation and end-to-end testing, without needing to build a custom binary with xk6.

All xk6-browser scripts that work with v0.8.0 will continue to work with the built-in module in k6 v0.43.0. To use them, change the import path from k6/x/browser to k6/experimental/browser, and set the environment variable K6_BROWSER_ENABLED to true. The requirement to specify the environment variable is temporary and may be removed in a future k6 release. It was added to minimize the risks with k6 unexpectedly launching a browser (or another process) from k6 scripts. It's also a mechanism we use in the k6 Cloud, where browser tests are currently disabled.

For details, review the script example, or the updated browser module documentation.

The module is currently under the experimental namespace, which means we reserve the decision to introduce breaking changes in the future. However, our mid-term goal is to drop the experimental label and make browser support a stable part of the k6 feature set, eventually enabling it in k6 Cloud as well.

Native support for JavaScript's async/await #2830

In v0.35.0 we added support for asynchronous functionality in k6 scripts with the addition of Promise.

While useful, the experience wasn't very friendly. Scripts had to use the .then() API to chain Promises, instead of the await syntax available in most other JavaScript runtimes, and the async keyword wasn't supported. Some workarounds were possible, but it required a separate build pipeline to transpile the syntax into the older ES5.1+ standard supported by k6.

That is, until now! 🎉 With invaluable help from @dop251, who maintains goja, the JS VM k6 uses, v0.43.0 brings native async/await to your k6 scripts. This functionality works just as you'd expect in other JS runtimes, and makes working with async APIs much more convenient. For details, review the following http.asyncRequest() example.

One caveat to note: async functions can't be passed to group() or check(). These functions are incompatible with asynchronous behavior, so you will get an error if trying to do so.

Experimental JavaScript module for distributed tracing #2853 #2854 #2855

This release brings a new experimental JavaScript module that adds distributed tracing support to k6. With one call in init context, you can instrument your test script's HTTP requests. If the system you're testing is instrumented in the same way, this module brings visibility to SUT behavior for the lifetime of each request.

An example:

import tracing from 'k6/experimental/tracing';
import http from 'k6/http';

tracing.instrumentHTTP({
  propagator: 'w3c',
});

export default () => {
  http.get('https://httpbin.test.k6.io/get', {
    headers: {
      'X-Example-Header': 'instrumented/get',
    },
  });
};

For details and examples, refer to the tracing module documentation.

http.asyncRequest #2877

The k6/http module has a new asyncRequest function that takes the same arguments as http.request(), but returns a Promise that, when used with await, will be resolved with a Response object. This gives you more control over script execution, as potentially the most time-consuming calls—making HTTP requests—will no longer block the thread of execution.

An example issuing a POST request:

import http from 'k6/http';

export default async function () {
  const resPromise = http.asyncRequest(
    'POST', 'https://httpbin.test.k6.io/post', { name: 'Bert' });
  // Do something else here, make other requests, etc.
  // Then when you're ready to use the response:
  const resp = await resPromise;
  console.log(resp.json().form.name); // Bert
}

This is one of the first steps towards migrating our APIs to be asynchronous, and similar changes can be expected in the future.

You can read more about asyncRequest in the documentation.

Enhancements and UX improvements

• #2754, #2805 The output of the k6 version command has been enhanced to also show the version of all extensions built into the k6 binary produced by xk6. Thanks, @HarrisChu!
• #2800 Improved handling of the Ctrl+C signal to gracefully abort the test during VU initialization.
• #2803 Ensure the REST API server is shut down after the test ends.
• #2867 Added the ability to display test run details and logs from k6 Cloud.
• #2890 Added a method for JavaScript modules to lookup environment variables without directly accessing the os package.
• #2910 Added a bit more context when parsing the script options, so that it is more obvious what fails.

Bug fixes

• #2829 The csv output now correctly shows vu and iter system tags. This fixes a regression introduced in v0.40.0. Thanks, @leonyork!
• #2851 Calling k6/execution.test.abort() within a group() now correctly exits the k6 process with code 108. Thanks for reporting this, @pomeh!
• #2896 Fixed a panic in k6/ws when using Socket.setInterval() with values between 0 and 1.
• #2903 Fixed a regression introduced in v0.42.0 where k6 will load the wrong module if the same import specifier has already been loaded, but they are pointing to different absolute paths, based on the files they are imported from.
• #2907 Fixed the exit code and run_status value when the cloud output aborts a test.

Maintenance and internal improvements

• #2809, #2810, #2812, #2813, #2815, #2885, #2893 A core component of test execution, the Engine, was removed, and the behavior heavily refactored. This change simplifies the code base and unblocks further improvements.
• #2821, #2864 Our high-level integration test suite was refactored and expanded, increasing the coverage of behavior that closely replicates real-world usage.
• #2803 Enabled checking for goroutine leaks.
• #2883 The goja runtime has been updated.
• #2845 Lint fixes in the k6/http module.
• #2831 Compatibility with TC39 error messages was improved.
• #2846 The initialization of js.Bundle was simplified.
• #2861, #2870 Some deprecated features and dependencies in our CI pipeline were removed.
• #2882 Our Dockerfile was improved with some linter suggestions. Thanks, @kempsterc!

Full Changelog: v0.42.0...v0.43.0

v0.42.0

k6 v0.42.0 is here! 🎉

This release includes:

• A tiny breaking change to improve WebSockets response handling.
• A new experimental output.
• More features in our experimental WebSocket module.
• Wildcard support for hosts.
• Some bug fixes, UX improvements, and maintenance.

Breaking changes

• #2712 k6/ws returns an HTTP response for failed connections instead of an undefined behavior. Thanks, @brietaylor.

New Features

Experimental Prometheus Remote Write Output #2784

This release brings a new builtin Output to any Prometheus Remote Write implementations (e.g. Prometheus, Mimir). This is an experimental feature, and future releases could introduce breaking changes.

The following example uses k6 run with the new output. It uses the defaults options, such as the Remote Write server URL (http://localhost:9090/api/v1/write):

k6 run -o experimental-prometheus-rw script.js

It supports the new and convenient experimental Native Histogram feature, added in Prometheus v2.40.0. It's not enabled by default, but we expect to make it the default way to map k6 Trend metrics once the Prometheus project signals that its mature enough and when more Remote Write implementations support it. For now, if you want to use it, you need to set the environment variable K6_PROMETHEUS_RW_TREND_AS_NATIVE_HISTOGRAM to true.

You can find complete documentation with more examples, use cases, and available configurations.

More features for the experimental websockets module #2786

The k6/experimental/websockets module that we announced in the v0.40.0 release got an update that extends its functionality.

It brings some useful features that the k6/ws module already has, like cookies, custom headers, compression and tags customization support, the syntax to define event handlers (onopen, onmessage, etc.) and ping/pong functionality.

This is still an experimental module, but with the recent changes we think it's usable for most users. So whether you're writing a new WebSocket test, or currently using the k6/ws module, we invite you to give it a try, and report any issues in the project's issue tracker.

Expand to see an example of the new WebSockets functionality

This example customizes tags for a WebSocket connection, sets up handlers using the new on* syntax, and demonstrates the ping/pong feature.

import { WebSocket } from "k6/experimental/websockets";
import {
  setTimeout,
  clearTimeout,
  setInterval,
  clearInterval
} from "k6/experimental/timers";

const CLOSED_STATE = 3

export default function () {
  const params = {
    "tags": {
      "my_tag": "hello"
    }
  };

  const ws = new WebSocket('ws://localhost:10000', null, params);

  ws.onopen = () => {
    console.log('connected');
    ws.send(Date.now().toString());
  };

  let intervalId = setInterval(() => {
    ws.ping();
    console.log("Pinging every 1 sec (setInterval test)");
  }, 1000);

  let timeout1id = setTimeout(function () {
    console.log('2 seconds passed, closing the socket');
    clearInterval(intervalId);
    ws.close();
  }, 2000);

  ws.onclose = () => {
    clearTimeout(timeout1id);
    console.log('disconnected');
  };

  ws.onping = () => {
    console.log("PING!");
  };

  ws.onpong = () => {
    console.log("PONG!");
  };

  // Multiple event handlers on the same event
  ws.addEventListener("pong", () => {
    console.log("OTHER PONG!");
  });

  ws.onmessage = (m) => {
    let parsed = parseInt(m.data, 10)
    if (Number.isNaN(parsed)) {
      console.log('Not a number received: ', m.data);
      return
    }

    console.log(`Roundtrip time: ${Date.now() - parsed} ms`);

    let timeoutId = setTimeout(function () {
      if (ws.readyState == CLOSED_STATE) {
        console.log("Socket closed, not sending anything");

        clearTimeout(timeoutId);
        return;
      }

      ws.send(Date.now().toString());
    }, 500);
  };

  ws.onerror = (e) => {
    if (e.error != "websocket: close sent") {
      console.log('An unexpected error occurred: ', e.error);
    }
  };
};

The module docs has a complete reference, and some examples.

Wildcard support for hosts option #2747

Thanks to the great effort from @eugercek, the hosts option now accepts domains that contain a wildcard at the beginning.
It can be helpful for setting multiple subdomains of the same domain, so instead of setting subdomain1.k6.io': '1.2.3.4', 'subdomain2.k6.io': '1.2.3.4' it is possible to use the wildcard for setting directly *.k6.io: '1.2.3.4'.

export const options = {
  hosts: {
    '*.k6.io': '1.2.3.4',
  },
}

Enhancements and UX improvements

• #2660 Pre-loads the operating system TLS certificates. Thanks, @tbourrely.
• #2791 Initializes VUs for setup() and teardown() only if they are defined in the script.

Bug fixes

• #2759 Ensures the evaluation of thresholds over trend metrics' median.
• #2759 Fixes a few potential Output data races for interrupted test runs.
• #2767 Fixes the emission of ws_session_duration when setup throws an error.
• #2773 Ensures that JavaScript runtime makes only one copy of the exports for each module including built-in ones.

Maintenance and internal improvements

We had a few minor changes in this release:

• #2757 goja runtime has been updated.
• #2768 WS.Connect() has been refactored.
• #2770 Refactored parts of the js module.
• #2782 Covered more relative path test cases for require and open.
• #2789, #2792, #2795, #2796 Improved stability of the integration tests.
• #2791 Optimized the performance of the internal Trie implementation. Thanks, @eugercek.

Full Changelog: v0.41.0...v0.42.0

v0.41.0

k6 v0.41.0 is here! 🎉 It has relatively few user-facing changes, but includes massive internal improvements that pave the way for some awesome features for the near future. Unfortunately, this work also required us to make a few minor breaking changes.

Breaking changes

Changes in the url, iter and vu system metric tags

As we warned in the release notes for k6 v0.39.0 and v0.40.0, we've been fundamentally refactoring the metrics sub-systems of k6. We now have efficient support for time series, which required a few minor user-facing breaking changes:

• If URL grouping is used for HTTP requests (that is, if the http.url helper is used or the name metric tag is specified), then the url tag in the resulting http_req_* metric samples will also have the same value as the name tag. Previously, k6 did this only for the cloud output, but now it does this universally (#2703).
• The vu and iter system tags, which are disabled by default, have been transformed into high-cardinality metrics metadata instead. It means that they will no longer be usable in thresholds, and various outputs may emit them differently or ignore them completely (#2726).

Changes in the Go metrics APIs

While the user-facing changes from our metrics refactoring are few and relatively minor, and there are no changes to JavaScript APIs yet, we have extensively refactored our internal Go APIs (#2594, #2726, #2727). The metrics.Sample and metrics.TagSet types are now entirely different. We also have high-cardinality metadata attributes in each Sample and at the VU level (see the combined TagsAndMeta code and how it is used in the per-VU State object).

k6 convert is officially deprecated (#2714)

k6 convert has been a sub-command to convert a HAR file recording of HTTP traffic into a preliminary k6 script that makes roughly the same requests. It has been long neglected and softly deprecated in favor of the newer and more feature-rich har-to-k6 standalone converter.

We have now officially deprecated k6 convert. The command still works and will continue to do so for a few more k6 versions. However, it's not visible from k6 --help and will emit a warning when used. Please see the documentation for the standalone har-to-k6 converter and open an issue (or comment on an existing one) if you have any problems with it.

New Features, enhancements, and UX improvements

• #2679 added support for maxReceiveSize and maxSendSize parameters in the gRPC's Client.connect() method. Thanks, @ariasmn!
• #2605 introduced a new --exclude-env-vars CLI flag to k6 archive that causes it to not include the provided environment variables in the resulting archive bundle's metadata.json file.
• #2700 added support for loading gRPC protoset files. Thanks, @jklipp!

Bug fixes

• #2678 fixed the Docker image labels. Thanks, @knittl, for reporting the problem (#2677)!
• #2689 fixed the REST API's Content-Type response header. Thanks, @wingyplus!
• #2691 fixed the detailed k6 version information embedded in the k6 releases.
• #2693 fixed a bug that made the k6 event loop unusable when a Promise rejection was left unhandled.
• #2696 fixed a problem with HTTP redirects with empty Location headers (#2474) by updating the Go version we use to compile k6 to 1.19.x. Thanks, @agilob!
• #2705 fixed a panic in the k6/net/grpc module (#2661). Thanks, @c47gao and @robpickerill!
• #2738 fixed a panic when a Promise was rejected with an undefined reason.
• #2739 fixed hidden stack traces in certain types of errors originating from k6's Go code.

Maintenance and internal improvements

We had a few minor changes in this release:

• #2687 improved our logging tests. Thanks, @nsmith5!
• #2696 updated the used Go version to 1.19.x and the Alpine version in our Docker image to 3.16. Thanks, @agilob!
• #2707, #2708, #2709, #2710 updated most of the Go dependencies k6 has.
• #2716 refactored how custom JS tags are applied to metrics and cleaned up validation for invalid tag values.

We also have a couple of significant improvements that will help us develop exciting new features soon:

Metric time series (#2594)

Previous to #2594, k6 didn't have an efficient way to group metric samples with the same tags. It meant that a whole class of applications for processing and aggregating metrics were nearly impossible to do or, at best, very inefficient.

At the cost of some minor breaking changes, we now have a performant internal representation to group metric samples with the same tags at the time of the action that generated them, i.e. the time of metric measurement. With this, k6 can efficiently group samples for the same action (e.g. an HTTP request to a specific URL) over time and construct time series with them.

Internal support for high-cardinality metrics metadata (#2726, #2727)

As described in the previous section, the efficient grouping of metric samples into time series works well for relatively low-cardinality data. However, k6 needed some way to attach high-cardinality metadata as well. This is necessary for data that's unique or random, such as Trace and Span IDs in distributed tracing or user IDs in tests with huge data sets.

k6 v0.41.0 has added support for attaching high-cardinality metadata to metric samples, and the vu and iter system tags have been transformed into such metadata (see the breaking changes section above), but it is not yet accessible from user scripts. There is no JavaScript API to modify this metadata, only built-in k6 Go modules and xk6 Go extensions can make use of it, for now.

v0.40.0

k6 v0.40.0 is here! This release includes:

• Breaking changes to some undocumented and unintentional edge behaviors.
• New experimental modules and first-class support for JavaScript classes.
• Bugs and refactorings to pave the way for future features.

Finally, the Roadmap goes over the plans for the next cycles.

Breaking changes

• #2632 During the refactoring to set tags to metric.add in the order they are provided, we discovered that you could provide tags as a key-value pair map multiple times in the same call. This was never the intended use and was never documented. As it was undocumented, and as such functionality makes no sense alongside every other API k6 has, we decided to remove this ability.
• #2582 [For extensions using the event loop] Previously, if RegisterCallback result was called twice, the second call would silently break the event loop. This has never been expected behavior, and calling it twice is always a bug in the code using it. Now, calling the RegisterCallback result twice leads to a panic.
• #2596 The tainted property of the Metric type is no longer outputted by the JSON output. That property was likely always going to have a false value as it was outputted at the beginning of the test.

Main module/script no longer pollutes the global scope #2571

During the ESM changes, we found that anything defined in the main module scope was also accessible globally. This was because it was directly evaluated in the global scope.
This has now been remedied and is no longer the case. This is a breaking change, but given that the whole point of modules (CommonJS or ESM) is to separate them, this is obviously rather a bug than a feature.

On that note, we've seen reports by people who have this global accessibility of the main module (intentionally or not). Still, it seems relatively rare, with only a few usages in a script. So if you need to access something globally, our suggested workaround is to set it explicitly on the global object globalThis.

k6/ws now respects the throw option #2247

k6/http has used the throw option to figure out whether it should throw an exception on errors or return a response object with an error set on it (and log it).

This functionality is finally also available for k6/ws, which previously would've always thrown an exception and thus involved more scripting in handling it (#2616).

This is a minor breaking change. By default, throw is false, so it now no longer throws an exception but instead returns a Response with error property.

Thank you, @fatelei, for making this change!

New Features

Experimental modules #2630 and #2656

As mentioned in the v0.39.0 release notes, we're happy to announce that this release brings experimental modules. The main idea behind this initiative is to get community feedback earlier, which will help us improve them faster. We encourage you to try experimental modules out and provide feedback through the community forums or GitHub issues.

This release contains three experimental modules:

• k6/experimental/redis - support for interaction with Redis
• k6/experimental/websockets - a new Websockets API that copies the "web" WebSocket API
• k6/experimental/timers - adding setTimeout/clearTimeout and setInterval/clearInterval implementations.

Important to highlight that the k6 team does not guarantee backward compatibility for these modules and may change or remove them altogether. Also, their import paths, starting with k6/experimental, will break when the modules stop being experimental. Of course, we are going to try to limit those breaking changes to a minimum and, when possible, do them in a backward compatible way for at least a version.

Redis example

Here is a fairly big example using xk6-redis as an experimental module to keep track of data in Redis:

import { check } from "k6";
import http from "k6/http";
import redis from "k6/experimental/redis"; // this will be `k6/x/redis` if you are using it as extension
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export const options = {
  scenarios: {
    usingRedisData: {
      executor: "shared-iterations",
      vus: 10,
      iterations: 200,
      exec: "measureUsingRedisData",
    },
  },
};

// Instantiate a new redis client
const redisClient = new redis.Client({
  addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
  password: __ENV.REDIS_PASSWORD || "",
});

// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);

export function setup() {
  redisClient.sadd("crocodile_ids", ...crocodileIDs);
}

export function measureUsingRedisData() {
  // Pick a random crocodile id from the dedicated redis set,
  // we have filled in setup().
  redisClient
    .srandmember("crocodile_ids")
    .then((randomID) => {
      const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
      const res = http.get(url);

      check(res, {
        "status is 200": (r) => r.status === 200,
        "content-type is application/json": (r) =>
          r.headers["Content-Type"] === "application/json",
      });

      return url;
    })
    .then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
}

export function teardown() {
  redisClient.del("crocodile_ids");
}

export function handleSummary(data) {
  redisClient
    .hgetall("k6_crocodile_fetched")
    .then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
    .then((data) =>
      redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
    )
    .then(() => redisClient.del("k6_crocodile_fetched"));

  return {
    stdout: textSummary(data, { indent: "  ", enableColors: true }),
  };
}

This example also showcases how to write some data and clean up after yourself.

The extension does not run a Redis server. You need to separately handle running, scaling, and connecting infrastructure to Redis.

The xk6-redis repository has more examples, and the module is documented in the official k6 documentation.

WebSockets example

This is a rewrite of the current WebSocket example at https://test-api.k6.io/.

This showcases how a single VU can run multiple WebSockets connections asynchronously and how to stop them after a period using the timeout and interval functions.

import { randomString, randomIntBetween } from "https://jslib.k6.io/k6-utils/1.1.0/index.js";
import { WebSocket } from "k6/experimental/websockets"
import { setTimeout, clearTimeout, setInterval, clearInterval } from "k6/experimental/timers"

let chatRoomName = 'publicRoom'; // choose your chat room name
let sessionDuration = randomIntBetween(5000, 60000); // user session between 5s and 1m


export default function() {
  for (let i = 0; i < 4; i++) {
    startWSWorker(i)
  }
}

function startWSWorker(id) {
  let url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;
  let ws = new WebSocket(url);
  ws.addEventListener("open", () => {
    ws.send(JSON.stringify({ 'event': 'SET_NAME', 'new_name': `Croc ${__VU}:${id}` }));

    ws.addEventListener("message", (e) => {
      let msg = JSON.parse(e.data);
      if (msg.event === 'CHAT_MSG') {
        console.log(`VU ${__VU}:${id} received: ${msg.user} says: ${msg.message}`)
      }
      else if (msg.event === 'ERROR') {
        console.error(`VU ${__VU}:${id} received:: ${msg.message}`)
      }
      else {
        console.log(`VU ${__VU}:${id} received unhandled message: ${msg.message}`)
      }
    })


    let intervalId = setInterval(() => {
      ws.send(JSON.stringify({ 'event': 'SAY', 'message': `I'm saying ${randomString(5)}` }));
    }, randomIntBetween(2000, 8000)); // say something every 2-8seconds


    let timeout1id = setTimeout(function() {
      clearInterval(intervalId)
      console.log(`VU ${__VU}:${id}: ${sessionDuration}ms passed, leaving the chat`);
      ws.send(JSON.stringify({ 'event': 'LEAVE' }));
    }, sessionDuration);

    let timeout2id = setTimeout(function() {
      console.log(`Closing the socket forcefully 3s after graceful LEAVE`);
      ws.close();
    }, sessionDuration + 3000);

    ws.addEventListener("close", () => {
      clearTimeout(timeout1id);
      clearTimeout(timeout2id);
      console.log(`VU ${__VU}:${id}: disconnected`);
    })
  });
}

Note that no k6 iterations finish if any WebSocket is still open or if a timeout or an interval is not cleared or triggered. This means that your script must take care of clearing all intervals and closing the WebSocket at some point. However, k6 still kills the whole process if it takes too long to stop after the maximum test duration is reached.

Current issues and ...

v0.39.0

k6 v0.39.0 is here! 🎉 It's a small release that includes a bunch of bugfixes and minor enhancements. Much of our focus was on some upcoming big changes. You can read about what's coming up next in the Roadmap and future plans section.

Enhancements and UX improvements

• #2274 and #2560 improved the csv output with support for a new timeFormat option. The possible values are unix (default) and rfc3399. You can also configure it through the K6_CSV_TIME_FORMAT environment variable. Thanks, @rpocklin!
• #2509 added the clear() and delete() methods to the CookieJar object from the k6/http module. Thanks, @Maksimall89!
• #2282 increased the precision of the iteration-progress bar in the UI. Thanks, @m3hm3t and @darkaether!
• #2568 added more descriptive error messages when there were problems with parsing a config file.

Bug fixes

• #2523 fixed a gRPC marshaling error when any.proto was used for a type. Thanks, @Flowersea!
• #2534 fixed the return type of Selection.map() from the k6/html module to the correct object types instead of a forced array of strings.
• #2502 made it so k6 waits for scenario executors to fully finish before updating their final progress in the UI, preventing misleading red crosses (#2500).
• #2524 fixed a bug where GoError string contained missing URL values (#2537).
• #2530 fixed a wrong error message on remote resolution.
• #2542 fixed a bug where Rate metric and sub-metric values were shown as NaN in the end-of-test summary if there were no measured values for them during the test run.
• #2558 fixed a panic when trying to set the value of a vu.tags element from k6/execution to null or undefined.
• #2567 fixed a panic when trying to access some k6/execution properties outside of a VU context, e.g. trying to access execution.scenario in setup().

Maintenance and internal improvements

• #2550 updated the used Go version to 1.18.
• #2524, #2551, #2552, #2553, #2554, #2555 updated various Go dependencies in k6.
• #2583 added a deprecation warning for thresholds that use the url, error, vu or iter tags, which will become un-indexable in the future.

Roadmap and future plans

As the lack of big changes in this release suggests, we've focused the last few months' efforts on a few areas that haven't yet been merged into the core of k6.

In this section, we'd like to inform the community about important features that we're currently working on - our short-term roadmap in a sense. We'll also use it to give notice of breaking changes we plan to make in the near future.

k6/experimental/* JS modules

Over the last several k6 releases, among a lot of other refactorings, we've added support for JavaScript event loops (#2228) in k6 VUs and added a new Go API for exposing built-in and xk6 extension modules to user scripts (announcement, docs). This has given us (and any xk6-extension authors!) the ability to better support various asynchronous streaming/messaging/etc. protocols (#882).

We've started building some of these newly possible APIs as xk6 extensions first, to be able to iterate on them more quickly and get some user feedback while we are building them. xk6-websockets, xk6-timers and xk6-redis are some of the first such APIs, but we plan to also work on support for gRPC streaming (#2020), messaging protocols (#1269), a new and better HTTP API (#2461) and many others in the future!

We want to eventually include a lot of these APIs in the k6 core as built-in modules that users can directly use, without needing to mess with xk6 or Go compilation. However, because we try to keep the built-in k6 APIs stable and backwards-compatible, we want to get more user feedback before we do that, while we are still free to iterate and make (hopefully minor) breaking changes.

So, we decided to create a new middle ground between the unstable and purely external xk6 extensions and the stable built-in k6 APIs―built-in k6/experimental/* modules! Our goal is that, starting with the next k6 v0.40.0 release, we'll start releasing some or all of these core-bound extensions as built-in k6 modules under these k6/experimental/ import paths. This will let k6 users, both OSS and Cloud, to give us feedback and help us improve the APIs before we stabilize them.

As is hopefully clear from the name, our usual guarantees of API stability won't apply to these modules while they are still experimental. We reserve the right to make breaking changes in experimental modules, up to and including completely dropping them. We don't expect big breaking changes will need to happen often, but we want to be clear they aren't impossible. Finally, when an API has been stabilized and made available under a regular import path, we'll deprecate its experimental import path. To make the transition easier, both import paths will be available simultaneously for at least one k6 version.

Native support for ECMAScript modules

At the moment, k6 has support for ECMAScript modules (ESM, i.e. import, export, etc.) via automatic transpilation of scripts by the built-in Babel.js. That mostly works, but it has caused some performance and compatibility problems (#824 and #2168 among others), so we want to support ESM modules and all other ES6 features directly in k6, without the need for Babel.js (#2296). goja, the JavaScript runtime we use to evaluate k6 scripts, doesn't yet have native ESM support, so we are currently working on adding it there, to then be able to support ECMAScript modules natively in k6!

That work has been ongoing for a while and we're making progress, but it will likely not be ready in time for the next k6 v0.40.0 release. We are mentioning it here because we will probably need to make a few minor breaking changes and fixes of currently undefined behavior in k6 to support ESM modules natively.

For example, at the moment, some values like the consolidated script options were unintentionally made available globally, in all JS modules of a test, instead of just in the exported options value from the main JS module. That is not the intended or documented behavior, it's somewhere between a bug and undefined behavior, and we'll need to fix it (#2571) and other similar issues like it, starting in k6 v0.40.0. We don't expect many scripts to break because of these fixes, but we'll nevertheless announce them in the release notes of the k6 version that they happen in.

Refactoring metrics

Over the last several k6 releases, we've also slowly been refactoring and improving the metrics internals in k6 (see #2071, #2330, #2426, #2463, #2433, #2442, among others). This has already brought many side benefits and minor bugfixes, and we still have a lot of work left (e.g. #1889, #572, #2430, #1831), but we've finally reached the point where we can almost start implementing some major new features effectively!

One of the upcoming next big steps is the introduction of a "time series" concept internally in k6 (#2580). We'll start to efficiently group samples (i.e. metric measurements) with the same metric and tags into a single TimeSeries, which ...

v0.38.3

k6 v0.38.3 is a patch release containing a single fix

Threshold over already defined sub-metrics will result in an error (#2538)

There was a bug where we were checking if a submetric had already been added. Unfortunately, we didn't check that this will work with the one submetric we have by default http_req_duration{expected_response:true}. After v0.38.0 defining a threshold on it would result in an error.

As this definitely shouldn't happen in that case and we don't see a particular case where that will be problematic - adding a submetric again just reuses the already added one instead.

This issue has been addressed in #2539, and k6 v0.38.3 will now lead you add a threshold on http_req_duration{expected_response:true}.

v0.38.2

k6 v0.38.2 is a patch release containing a couple of bugfixes!

Threshold over sub-metrics without samples would result in NaN (#2520)

There was a bug in thresholds applied to sub-metrics set to abortOnFail: leading k6 to evaluate thresholds that would have likely aborted before they had a chance of passing (because no samples for the given metric were recorded yet). This bug would have led to such thresholds' results value to be NaN rather than a numerical value. The following script, for instance:

import { check, sleep } from 'k6';

import http from 'k6/http';

export const options = {
  scenarios: {
    iWillFail: {
      exec: 'iWillFail',
      executor: 'constant-vus',
      startTime: '2s',
      vus: 1,
      duration: '30s',
    },
  },

  thresholds: {
    'checks{type:read}': [{ threshold: 'rate>0.9', abortOnFail: true }],
  },
};

export function iWillFail() {
  let res = http.get(`https://test-api.k6.io/`);

  check(res, {
    'read status is 200': (r) => r.status === 200,
  }, { type: 'read' });

  sleep(1);
}

Would result in the following:

✗ { type:read }...: NaN% ✓ 0 ✗ 0  
vus...............: 0 min=0 max=0
vus_max...........: 1 min=1 max=1

This issue was introduced by recent changes to how we handle thresholds in the k6 engine and is now addressed in v0.38.2.

Sub-metrics without values rendered below an incorrect parent metric (#2518)

There was in how thresholds over sub-metrics that didn't receive any samples would be displayed under an incorrect parent metric. For instance, the following script:

import { Counter } from 'k6/metrics';

const counter1 = new Counter("one");
const counter2 = new Counter("two");

export const options = {
    thresholds: {
        'one{tag:xyz}': [],
    },
};

export default function() {
    console.log('not submitting metric1');
    counter2.add(42);
}

Would have led to the following output, where the {tag:xyz} sub-metric is displayed under iterations instead of one:

data_received........: 0 B 0 B/s
data_sent............: 0 B 0 B/s
iteration_duration...: avg=0s min=0s med=0s max=0s p(90)=0s p(95)=0s
iterations...........: 1 499.950005/s
  { tag:xyz }........: 0 0/s
two..................: 42 20997.90021/s

When we would have expected it to produce:

one..................: 0 0/s
  { tag:xyz }........: 0 0/s
two..................: 42

This issue has been addressed in #2519, and k6 v0.38.2 now displays sub-metrics under their actual parents, even when they have received no samples.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.

v0.38.1

k6 v0.38.1 is a patch release containing a bugfix!

Threshold sub-metric selectors containing reserved symbols would fail (#2512)

There was a bug in threshold sub-metric selector parsing, which led to errors when users would use specific symbols such as {, } or : as part of their definitions. For instance, thresholds used for sub-metrics with URL Grouping like http_req_duration{name:"http://example.com/${}"} would have led to failures in v0.38.0.

The error messages for invalid metric, sub-metric and threshold definitions were also improved.

Special thanks to @efdknittlfrank, who reported and helped us track down the issue.