jdx/hk v2.3.1

v2.3.1: Faster hook startup and fix runs, safer stashing

on GitHub

hk now starts faster, spends less of its own time per file, and spreads --jobs more evenly across batched steps. This release also fixes several cases where stashing could lose files, fail before any step ran, or leave untracked files visible to steps.

Added

• apply_check_diff step setting (#1495, @jdx). Defaults to true. If you set it to false, hk fix runs the step's fix command instead of applying the check_diff output, and hk check still shows the diff. In staging hooks such as pre-commit, hk still runs check_diff first and passes fix only the files the diff names. The trailing_whitespace and newlines builtins now use it. Together with a parallel file scan in hk util trailing-whitespace and hk util end-of-file-fixer, this roughly halves the time of those two steps on a whole-repo fix.

  ["my-fixer"] {
    check_diff = "my-fixer --diff {{files}}"
    fix = "my-fixer --write {{files}}"
    apply_check_diff = false
  }

Changed

• Every command starts sooner. hk now reads the config cache files in one call instead of one byte at a time, which cuts about 30 ms from every command that loads a config (#1483). Hooks read the repository status with a single git status --porcelain=v2, which saves about 10 ms more (#1484). The Linux GNU release binaries are now linked non-PIE, so each launch starts about 0.8 ms faster (#1482). The tradeoff is that ASLR no longer covers hk's own code and data. The heap, stack and shared libraries are still randomized. musl, macOS, Windows and cargo install builds are unchanged. (@jdx)

• Batched steps share --jobs (#1496, @jdx). Before, every step with batch = true in a group started up to --jobs processes on its own. With 8 jobs, the benchmark ran 40 processes for 8 slots. Batched steps that run at the same time now split the jobs between them: each step gets an equal share first, and leftover jobs go to the steps with the most files. A group with a single batched step still gets every job, and steps with depends or condition keep every job. On the benchmark, hk check --all used about 20% less CPU. Nothing needs to be configured.

• Fix runs are faster and more consistent.

  • A fixer no longer waits behind a job that shares none of its files. hk now gives a job all of its file locks at once or none of them. On the benchmark, the spread in hk fix --all fell from ±207 ms to ±30 ms (#1503).
  • Steps that finish together stage their fixes with one git add (#1485).
  • hk no longer hashes the whole git index before each step stages its fixes (#1486).
  • hk does less work per file before a step starts: faster file locks, parallel type, binary and symlink checks, and one lstat per file (#1494).
  • hk parses check_diff output without canonicalizing every file (#1487). If allow_symlinks puts both a symlink and its target in the same job, a diff that names one of them now selects only that path.

  (@jdx)

• jq and yq builtins check already-formatted files in one batch (#1489, @jdx). Before, they started the tool and diff once per file. On the benchmark, the jq check step went from 736 ms to 52 ms. Files that can't be confirmed as formatted this way still get the old per-file check, so patches, exit codes and error messages stay the same.

• Small changes to git.* status lists (#1484, @jdx). These lists now match git's own status in three edge cases:

  • Sparse-checkout entries missing from the worktree are no longer listed in git.unstaged_deleted_files.
  • An intent-to-add file that was then deleted is no longer listed in git.staged_added_files.
  • With HK_STASH_UNTRACKED=false, a submodule whose only change is untracked files no longer counts as unstaged.

• Building from source now requires Rust 1.94 (#1502, @jdx). This includes cargo install hk. Prebuilt binaries are unaffected.

Fixed

• Stashing no longer loses files when a filename is not valid UTF-8 (#1488, @jdx). With stash = "git", one such name could make hk restore nothing and then drop the stash, deleting untracked and unstaged files. hk now skips these paths with a warning and still stashes and restores them.
  • Restoring now uses the stash commits directly. This fixes two gaps: a staged edit later reverted to HEAD, and an unstaged deletion stashed together with untracked files.
  • When a step changed a path the stash also changed, hk keeps the stash and prints a git restore command for each path it didn't restore.
• Hooks run when the index has intent-to-add files (#1492, @jdx). With stashing on, a git add -N entry made hk run pre-commit and git commit fail with Entry '...' not uptodate. Cannot merge.. hk now saves these files in a separate stash entry, hides them from steps, and restores them afterwards as intent-to-add. If hk is killed during a hook, the entry appears as hk: intent-to-add files in git stash list.
• Untracked files are hidden from steps when tracked files also have unstaged changes (#1501, @jdx). Before, in that case, steps could read, overwrite or stage untracked files. hk now stashes the whole worktree whenever there are untracked files to stash.
• git commit works in repositories with core.splitIndex = true (#1491, @jdx). Before, the hook failed with failed to get index because libgit2 can't read a split index. hk now falls back to shell git in these repositories.
• Amended steps whose tests use a local named like a StepTest property now load (#1490, @jdx). Before, a local named, for example, after or files failed with invalid type: null, expected a string once the step was amended. hk now uses pklr 3.0.1, which fixes this.

Full Changelog: v2.3.0...v2.3.1

💚 Sponsor hk

hk is built and maintained by @jdx, an open source developer at entire.io, the title sponsor of his open source work.

If hk speeds up your pre-commit loop or makes linting less painful, please consider becoming an individual or company sponsor. Your support funds ongoing development and helps keep hk fast, free, and independent.