sharkdp/hyperfine v2.0.0

on GitHub

See the breaking changes below when migrating from hyperfine 1.x.

Features

• Add support for alternative performance metrics (peak memory usage, instructions, CPU cycles, cache misses, branch misses, ...). By default, hyperfine will now display wall-clock time and peak memory usage, but users can use the new --metrics option to select different performance metrics. The terminal output now shows an overview of all selected performance metrics (see here for a run with more metrics enabled).

  

• Add --env to set environment variables even without an intermediate shell (which is the new default behavior, see below). For example:

  # Parameterized benchmark, comparing different thread counts:
  hyperfine -P threads 1 8 --env 'OMP_NUM_THREADS={threads}' 'my_command'

• The plotting and analysis scripts now support different metrics as well, see #981 and #984 (@sharkdp).

Breaking changes

• Benchmarked commands are now executed directly by default, without an intermediate shell (--shell=none). Use -S (an alias for --shell=default) to restore the previous behavior (sh on Unix, cmd.exe on Windows), or select a shell with --shell <SHELL>. See the rationale for this change.

• Always run --setup, --prepare, --conclude, and --cleanup commands in a shell, even when using --shell=none, see #990 (@sharkdp).

• The JSON format for --export-json has changed (schema version 2). The new format now includes metadata, per-run measurements, and statistical summaries for all measured quantities, see #790 (@sharkdp). Code that reads those exported JSON files must be updated. For example, results[i].mean is now results[i].summary.time_wall_clock.mean, and results[i].times is replaced by results[i].measurements[j].time_wall_clock.value. The new structure looks like this:

  {
      "schema_version": 2,
      "primary_metric": "time_wall_clock",
      "metadata": {
          "hyperfine_version": "2.0.0",
          "start_time": "2026-10-06T12:00:00Z",
          "platform": {"os": "Linux", "architecture": "x86_64"}
      },
      "results": [
          {
              "command": "sleep 1",  # Actual command after parameter substitution
              "name": "wait 1s",  # Optional custom or automatically generated name
              "parameters": {"duration": {"value": "1"}},  # For parametrized runs
              "environment": {"OMP_NUM_THREADS": "8"},  # Only present with --env overrides
              "measurements": [  # One entry per run, excluding warmups
                  {
                      "time_wall_clock": {"value": 1.0, "unit": "second"},
                      "time_cpu": {"value": 0.0, "unit": "second"},
                      "time_user": {"value": 0.0, "unit": "second"},
                      "time_system": {"value": 0.0, "unit": "second"},
                      "memory_peak_resident": {"value": 1048576.0, "unit": "byte"},
                      "cpu_cycles": {"value": 5000000},  # Hardware counters, when available
                      "instructions": {"value": 10000000},
                      "exit_code": 0
                  },
                  ...  # Further runs omitted
              ],
              "summary": {
                  "time_wall_clock": {
                      "unit": "second",
                      "count": 30,
                      "mean": 1.0,
                      "stddev": 0.0,
                      "median": 1.0,
                      "min": 1.0,
                      "max": 1.0
                  }
                  # Other metrics have the same summary structure.
              }
          }
      ]
  }

  Times are always exported in seconds and memory in bytes. Hardware counters and their summaries have no unit field. Unavailable memory measurements and hardware counters are omitted; other available counters may also be included. stddev is null for a single run.

• --time-unit/-u has been removed. Units can now be selected using the --metrics METRIC[:UNIT],… option, e.g. --metrics time_wall_clock:ms,memory_peak_resident:MiB.

• --sort has been removed. It complicated hyperfine significantly and doesn't make too much sense with the new output format.

• --reference and --reference-name have been removed (for now). The first command is always considered as the reference command. Invocations using --reference now show migration guidance to put the reference command first.

• The format of Markdown, AsciiDoc and org-mode, and CSV exports has also been changed. Markdown, AsciiDoc, and org-mode contain only the primary metric (the first metric selected by --metrics). CSV exports all selected metrics side by side.