grafana/k6 v0.40.0

on GitHub

k6 v0.40.0 is here! This release includes:

• Breaking changes to some undocumented and unintentional edge behaviors.
• New experimental modules and first-class support for JavaScript classes.
• Bugs and refactorings to pave the way for future features.

Finally, the Roadmap goes over the plans for the next cycles.

Breaking changes

• #2632 During the refactoring to set tags to metric.add in the order they are provided, we discovered that you could provide tags as a key-value pair map multiple times in the same call. This was never the intended use and was never documented. As it was undocumented, and as such functionality makes no sense alongside every other API k6 has, we decided to remove this ability.
• #2582 [For extensions using the event loop] Previously, if RegisterCallback result was called twice, the second call would silently break the event loop. This has never been expected behavior, and calling it twice is always a bug in the code using it. Now, calling the RegisterCallback result twice leads to a panic.
• #2596 The tainted property of the Metric type is no longer outputted by the JSON output. That property was likely always going to have a false value as it was outputted at the beginning of the test.

Main module/script no longer pollutes the global scope #2571

During the ESM changes, we found that anything defined in the main module scope was also accessible globally. This was because it was directly evaluated in the global scope.
This has now been remedied and is no longer the case. This is a breaking change, but given that the whole point of modules (CommonJS or ESM) is to separate them, this is obviously rather a bug than a feature.

On that note, we've seen reports by people who have this global accessibility of the main module (intentionally or not). Still, it seems relatively rare, with only a few usages in a script. So if you need to access something globally, our suggested workaround is to set it explicitly on the global object globalThis.

k6/ws now respects the throw option #2247

k6/http has used the throw option to figure out whether it should throw an exception on errors or return a response object with an error set on it (and log it).

This functionality is finally also available for k6/ws, which previously would've always thrown an exception and thus involved more scripting in handling it (#2616).

This is a minor breaking change. By default, throw is false, so it now no longer throws an exception but instead returns a Response with error property.

Thank you, @fatelei, for making this change!

New Features

Experimental modules #2630 and #2656

As mentioned in the v0.39.0 release notes, we're happy to announce that this release brings experimental modules. The main idea behind this initiative is to get community feedback earlier, which will help us improve them faster. We encourage you to try experimental modules out and provide feedback through the community forums or GitHub issues.

This release contains three experimental modules:

• k6/experimental/redis - support for interaction with Redis
• k6/experimental/websockets - a new Websockets API that copies the "web" WebSocket API
• k6/experimental/timers - adding setTimeout/clearTimeout and setInterval/clearInterval implementations.

Important to highlight that the k6 team does not guarantee backward compatibility for these modules and may change or remove them altogether. Also, their import paths, starting with k6/experimental, will break when the modules stop being experimental. Of course, we are going to try to limit those breaking changes to a minimum and, when possible, do them in a backward compatible way for at least a version.

import { check } from "k6";
import http from "k6/http";
import redis from "k6/experimental/redis"; // this will be `k6/x/redis` if you are using it as extension
import { textSummary } from "https://jslib.k6.io/k6-summary/0.0.1/index.js";

export const options = {
  scenarios: {
    usingRedisData: {
      executor: "shared-iterations",
      vus: 10,
      iterations: 200,
      exec: "measureUsingRedisData",
    },
  },
};

// Instantiate a new redis client
const redisClient = new redis.Client({
  addrs: __ENV.REDIS_ADDRS.split(",") || new Array("localhost:6379"), // in the form of "host:port", separated by commas
  password: __ENV.REDIS_PASSWORD || "",
});

// Prepare an array of crocodile ids for later use
// in the context of the measureUsingRedisData function.
const crocodileIDs = new Array(0, 1, 2, 3, 4, 5, 6, 7, 8, 9);

export function setup() {
  redisClient.sadd("crocodile_ids", ...crocodileIDs);
}

export function measureUsingRedisData() {
  // Pick a random crocodile id from the dedicated redis set,
  // we have filled in setup().
  redisClient
    .srandmember("crocodile_ids")
    .then((randomID) => {
      const url = `https://test-api.k6.io/public/crocodiles/${randomID}`;
      const res = http.get(url);

      check(res, {
        "status is 200": (r) => r.status === 200,
        "content-type is application/json": (r) =>
          r.headers["Content-Type"] === "application/json",
      });

      return url;
    })
    .then((url) => redisClient.hincrby("k6_crocodile_fetched", url, 1));
}

export function teardown() {
  redisClient.del("crocodile_ids");
}

export function handleSummary(data) {
  redisClient
    .hgetall("k6_crocodile_fetched")
    .then((fetched) => Object.assign(data, { k6_crocodile_fetched: fetched }))
    .then((data) =>
      redisClient.set(`k6_report_${Date.now()}`, JSON.stringify(data))
    )
    .then(() => redisClient.del("k6_crocodile_fetched"));

  return {
    stdout: textSummary(data, { indent: "  ", enableColors: true }),
  };
}

The extension does not run a Redis server. You need to separately handle running, scaling, and connecting infrastructure to Redis.

The xk6-redis repository has more examples, and the module is documented in the official k6 documentation.

import { randomString, randomIntBetween } from "https://jslib.k6.io/k6-utils/1.1.0/index.js";
import { WebSocket } from "k6/experimental/websockets"
import { setTimeout, clearTimeout, setInterval, clearInterval } from "k6/experimental/timers"

let chatRoomName = 'publicRoom'; // choose your chat room name
let sessionDuration = randomIntBetween(5000, 60000); // user session between 5s and 1m


export default function() {
  for (let i = 0; i < 4; i++) {
    startWSWorker(i)
  }
}

function startWSWorker(id) {
  let url = `wss://test-api.k6.io/ws/crocochat/${chatRoomName}/`;
  let ws = new WebSocket(url);
  ws.addEventListener("open", () => {
    ws.send(JSON.stringify({ 'event': 'SET_NAME', 'new_name': `Croc ${__VU}:${id}` }));

    ws.addEventListener("message", (e) => {
      let msg = JSON.parse(e.data);
      if (msg.event === 'CHAT_MSG') {
        console.log(`VU ${__VU}:${id} received: ${msg.user} says: ${msg.message}`)
      }
      else if (msg.event === 'ERROR') {
        console.error(`VU ${__VU}:${id} received:: ${msg.message}`)
      }
      else {
        console.log(`VU ${__VU}:${id} received unhandled message: ${msg.message}`)
      }
    })


    let intervalId = setInterval(() => {
      ws.send(JSON.stringify({ 'event': 'SAY', 'message': `I'm saying ${randomString(5)}` }));
    }, randomIntBetween(2000, 8000)); // say something every 2-8seconds


    let timeout1id = setTimeout(function() {
      clearInterval(intervalId)
      console.log(`VU ${__VU}:${id}: ${sessionDuration}ms passed, leaving the chat`);
      ws.send(JSON.stringify({ 'event': 'LEAVE' }));
    }, sessionDuration);

    let timeout2id = setTimeout(function() {
      console.log(`Closing the socket forcefully 3s after graceful LEAVE`);
      ws.close();
    }, sessionDuration + 3000);

    ws.addEventListener("close", () => {
      clearTimeout(timeout1id);
      clearTimeout(timeout2id);
      console.log(`VU ${__VU}:${id}: disconnected`);
    })
  });
}

Note that no k6 iterations finish if any WebSocket is still open or if a timeout or an interval is not cleared or triggered. This means that your script must take care of clearing all intervals and closing the WebSocket at some point. However, k6 still kills the whole process if it takes too long to stop after the maximum test duration is reached.

Current issues and future improvements for the WebSockets API can be found in its issue tracker. Currently, documentation is available through MDN, though some features are not yet supported:

• no Blob binary type - ArrayBuffer is the default
• no onMessage and co. - only addEventListener

First-class support for JavaScript Classes

As part of updating goja, k6 got native support for classes. Again, that's native, as in not by transpiling by the internal Babel.

Because this actually implements classes as described in the latest ECMAScript specification, this also means we get a ton of additional class features that were never previously supported (for example, private fields). Additionally, at least one bug #1763 was fixed as a result of this, but probably many more as well.

Due to this fairly significant change, some code could behave differently. Please report any issues, though consider that it's possible that the new behavior is just the correct one.

Other updates from goja are:

• optimizations around using strings and some access patterns
• support for \u{01234} Unicode point syntax in regexp
• Fixed a case where interrupting the VM did not work, especially around try/catch usage (#2600). This was particularly problematic for k6, as it could lead to k6 hanging.

Many thanks to @dop251 for continuing to improve goja!

New Test runtime for module extension developers #2598

While we develop extensions internally, we often need to repeatedly create the same structure. With the addition of the event loops, it is now required to set it up as well. Even k6 team members get parts of this wrong every once in a while, so we added a small type to be used by (extension) module developers to write tests easier (#2602).

This API will likely change and evolve as we add more functionality or as we change the k6 internal API.

Bug fixes

• #2585 http.batch() now displays an error if it is not given exactly 1 argument(#1289). Thanks, @vanshaj!
• #2596 Fixes a potential data race in the JSON output. Includes a breaking change where tainted property is no longer outputted. That property was (likely) always going to have the value false as it was outputted at the beginning of the test.
• #2604 Fixes SSL keylogger not working with absolute paths.
• #2637 Fixes setting the options rps to 0 or below leading to exceptions. Now setting it to 0 or below disables the limit. Thanks, @tbourrely. #2613
• #2278 Reading options.tags directly was not possible. This was fixed by accident by #2631. k6/execution is still the recommended way to access the final options of the test.

Maintenance and internal improvements

• #2590 Updates direct dependencies without any interesting changes apart goja.
• #2591 Changes to the CI process to always build rpm/deb and windows packages and use nfpm to do it.
• #2593 Internal cleanup after finally removing common.Bind.
• #2597 Fixes go benchmarks we have broken over time.
• #2599 Reformats //nolint comments as part of getting ready for go 1.19.
• A bunch of fixes for tests #2589, #2620, #2625, #2643, #2647, #2648,
• #2607 Fixes the build badge in the README. Thanks @AetherUnbound!
• #2614 Fixes advice for RPM install on Amazon Linux.
• #2615 Improves documentation of the RegisterCallback, following feedback on how hard it was to understand.
• #2627 Create distinct test state objects for the pre-init and run phases.
• #2635 Drop License header in each file.
• #2636 Add SECURITY.md with instructions how to report security issues responsibly.
• #2641 Fix spelling of lose. Thanks @spazm!
• Update to golangci-lint v1.47.2 and enable a bunch more linters. #2609, #2611. Also, drop obsolete configurations #2619.

Roadmap and future plans

This section discusses our plans for future versions. Notice that two big ticket items are here again―ESM modules and metric refactoring. They remain on the roadmap mostly for the sheer size of the work required on both, but also for some unforeseen needed changes, which actually should make them better in the long run. It also so happens that it is vacation season so the k6 team rightfully is taking some personal time away.

Native support for ECMAScript modules

Native ESM support is coming. A PR to k6 is under development, and there's a branch that will become a PR to goja to add the support there. The k6 team is hopeful that this will land in the next version, v0.41.0!

It turned out that there were a lot more things to be added as functionality - dynamic import, and the tc39 group released the latest ECMAScript specification adding support of top-level await in modules. While neither k6 nor goja has support for the async/await syntax, yet, this changes significantly the internal of the change, which did require a not insignificant refactor.

Additionally, as previously mentioned, there were a bunch of changes to goja, including adding class support which also needed to be integrated.

A future breaking change is that using CommonJS style exports along import/export syntax in the same file will no longer be possible.

import http from "k6/http"

exports.default = function() {} // this will start to error out

It will still be possible to import a CommonJS module and require an ES module or use require in an ES module. Having a dependency cycle mixing and matching CommonJS and ESM is also unlikely to work properly, but might do so in particular cases.

This is really expected to have never been done by anyone as there isn't really much of a reason for this, but it is currently supported due to Babel transpiling everything to CommonJS behind the scenes.

Refactoring metrics

The refactoring of metrics is underway, with a PR for more performant internal representation. Unfortunately, this took longer to get to a stable state, and given the size and the type of change, the k6 team decided to hold merging it until very near the end of the cycle. It also doesn't have any noticeable change for most users. Instead, it will be merged early in the v0.41.0 release cycle, and then more changes to use it will be made through the release cycle.

Some of those changes include supporting non-indexable tags, which also have a read PR. This change is also essential for historical reasons connected with how the name tag works. As such, it also needed to be merged to release the internal metric refactor.

Future breaking change: as part of the many changes and experiments, we found out that we can keep url as an indexable tag. Previously the plan was that it along vu and iter will become non-indexable. However, the url tag is heavily used and enabled by default. Because of that (and other internal reasons), the new change will be that url will stay indexable, but if used with name will be overwritten to have the same value as name. Arguably this is what most users would want in the case when they are using name. We plan to add a non indexable raw_url tag for those that do not. As such, we no longer will be printing a warning when url is used in thresholds.

Even so, we did make a bunch of changes to the internals of k6 that will pave the way forward (#2629, #2631).

We are also still working on incorporating the newly developed time series data model for the Prometheus remote-write output extension. We are fixing bugs and improving the extension with the goal of eventually integrating it as a core built-in k6 output module in a future k6 release.

Distributed tracing support itself needs non-indexable tag support. Once that is merged, more work in that direction will be started.