jdx/hk v2.3.0

v2.3.0: Faster staging hooks and fix runs, plus batch_min_files

on GitHub

Staging hooks such as pre-commit now stage each fixer's files as soon as that fixer finishes, and hk fix --all makes better use of --jobs. This release also adds a batch_min_files step setting and fixes the jq/yq builtins, stashing before the first commit, and an intermittent git crash during concurrent staging.

Added

• batch_min_files for batched steps (#1475, @jdx). Steps with batch = true can set the minimum number of files hk passes to one process. The default is still 4. Raising it means fewer processes, which helps tools with slow startup, such as prettier and eslint, on whole-tree runs. In hk's benchmark, batch_min_files = 200 on prettier and eslint cut hk check --all from 1.56 s to 1.33 s. It made a small pre-commit a little slower, so the builtins' defaults are unchanged. OS command-length limits can still split a batch below this number.

  steps {
    ["prettier"] = (Builtins.prettier) {
      batch_min_files = 200
    }
  }

Changed

• Staging hooks are faster (#1466, @jdx). Before, each fixer waited for every other step to finish before staging its changes. Now each step stages its own files as soon as nothing else is using them. Steps with check_list_files or check_diff (such as builtin black and prettier) also stopped running their check before the fix in staging hooks. They still check first when one of their files has unstaged changes that hk didn't stash. Explicit stage = "<JOB_FILES>" and check_first = true work as before. Staging now always goes through git add, including with libgit2. git add reads paths from stdin, so very large fixes can't go over the command-line length limit, and it applies clean filters such as git-lfs. On the benchmark's 62-file commit, hk run pre-commit went from about 0.89 s to 0.45 s.
• hk fix doesn't hold job slots while waiting on file locks (#1473, @jdx). A fixer waiting on another fixer's files now gives up its --jobs slot while it waits, so steps that work on other files can run in the meantime. On the benchmark project, hk fix --all went from 4.72 s to 4.08 s with the same output.
• More release binaries use profile-guided optimization (#1470, @jdx). The x86_64 Linux musl and macOS arm64 builds are now built with PGO, which only the x86_64 Linux glibc build had before. Archive names and layout are unchanged.

Fixed

• jq and yq builtins apply their own patches (#1468, @jdx). The patches from check_diff had a bad file label, so git apply rejected them and hk ran the tool again. hk now applies the patch directly, which roughly halves the time for these steps.
  • If a file can't be parsed (for example, JSONC or broken YAML), hk leaves it unchanged instead of emptying it, and hk fix exits non-zero.
  • jq's fix now keeps file permissions and fails when jq fails.
  • jq's check now sorts keys the same way its fix does, so hk check now fails on JSON files with unsorted keys that it used to pass.
• Stashing in a repository with no commits yet (#1469, @jdx). With the git CLI backend (HK_LIBGIT2=0), a hook that stashes unstaged changes failed in a repository with no commits because git stash errored. hk now skips the stash when HEAD doesn't exist, as the libgit2 backend already did.
• Intermittent "terminated by signal" failures during staging (#1476, @jdx). In a staging hook, git status or git add could crash with SIGBUS while another step was rewriting a different file, and could sometimes leave behind a stale index.lock. hk now runs git status with GIT_OPTIONAL_LOCKS=0. Before staging, it also locks any recently changed files that git might re-read while writing the index.

Full Changelog: v2.2.0...v2.3.0

💚 Sponsor hk

hk is built and maintained by @jdx, an open source developer at entire.io, the title sponsor of his open source work.

If hk speeds up your pre-commit loop or makes linting less painful, please consider becoming an individual or company sponsor. Your support funds ongoing development and helps keep hk fast, free, and independent.