github yamadashy/repomix v1.14.0

10 hours ago

This release is a major performance overhaul — packing the Repomix repository now takes about 1.4 seconds (down from 3.3 seconds in v1.13.1) — roughly 2.4× faster, a 58% reduction. Faster startup, lighter dependencies, and a smarter pipeline that overlaps work across stages.

What's New 🚀

Monorepo-Aware Tech Stack Detection (#1310, #1317)

The skill generator (--skill-generate) now detects dependency files in subdirectories and groups results by package directory. Monorepos with packages under packages/*/package.json, apps/*/package.json, etc. now produce a tech-stacks.md with a separate section per workspace, each listing its own languages, frameworks, dependencies, and runtime versions. Previously only the root-level dependency file was inspected.

Improvements ⚡

The 58% pack-time reduction is the cumulative result of dozens of optimizations across startup, the pipeline, worker IPC, and remote downloads — no single change accounts for the full speedup. The most impactful changes are highlighted below.

image

Replaced tiktoken WASM with gpt-tokenizer (#1350)

Token counting now uses gpt-tokenizer, a pure-JavaScript tokenizer, in place of the previous WASM-based tiktoken. This eliminates ~200 ms of WASM initialization overhead from startup and works in environments where WASM is restricted. Token counts are preserved — gpt-tokenizer is configured to match tiktoken's default behavior.

Eliminated Child Process in Default Action (#1372)

The default repomix action no longer spawns a child process for the main pack flow. This removes process startup overhead — most noticeable on smaller repositories where startup was a meaningful fraction of total time.

Wrapper-Extraction Fast Path for Token Counting (-13.2%) (#1457)

For non-parsable XML/Markdown/Plain output, Repomix now reuses per-file token counts and tokenizes only the output "wrapper" (header, separators, footer) instead of re-tokenizing the entire ~MB-scale output. This delivers a ~13% reduction in total pack time on typical repositories.

Pipeline Parallelization

The pack pipeline now overlaps stages that don't depend on each other:

  • Security check and file processing run concurrently (#1359)
  • Output generation overlaps with metrics calculation (#1359)
  • Git sort data is prefetched alongside file search and collection (#1467)
  • Wrapper tokenization runs in parallel with file metrics (#1469)
  • CLI actions are lazy-loaded so each command imports only what it needs (#1346)

Faster Startup

  • Removed Zod from the startup path (#1306)
  • Lazy-loaded handlebars, fast-xml-builder, and @clack/prompts (#1436)
  • Lazy-loaded jschardet and iconv-lite for encoding detection (#1401)
  • Removed gpt-tokenizer from the config schema's import chain (#1500)
  • Skipped the worker pool when only lightweight transforms are needed (#1338)
  • Eliminated a redundant stat() syscall in file reading (#1400)

Worker & IPC Optimizations

  • Batched token counting IPC (#1411)
  • Batched security check tasks (#1380)
  • Warmed up metrics worker threads in parallel (#1374)
  • Capped security worker threads at 2 to reduce contention (#1409)
  • Cached empty-directory paths across pipeline stages (#1356)
  • Combined file and directory globby walks into a single traversal (#1506)

Faster Remote Repository Downloads

  • Used codeload.github.com URLs directly to skip the 302 redirect (#1375)
  • Skipped binary files during tar extraction (#1392)

@secretlint/profiler Overhead Removed (-6.5%)

Disabling the profiler in the security worker reduced pack time by ~6.5% (#1453). A follow-up patch to perf_hooks.performance.mark handles duplicate @secretlint/profiler singletons that survive across hoisted/nested copies (#1456).

How to Update

npm update -g repomix

As always, if you have any issues or suggestions, please let us know on GitHub issues or our Discord community.

Don't miss a new repomix release

NewReleases is sending notifications on new releases.