uutils/coreutils 0.3.0

on GitHub

📦 Rust Coreutils 0.3.0 Release:

We are happy to announce the release of Rust Coreutils 0.3.0 — our latest milestone bringing significant performance improvements, enhanced compatibility, and robust testing infrastructure!

Highlights:

• Performance Improvements
  Compared to GNU:

  • base64 is 1.20x faster
  • expand is 1.80x faster
  • unexpand is 1.50x faster
  • nl is 1.57x faster
  • sort is 3.72x faster for regular sorting and 1.46x faster for numeric sorting
  • fold is 1.19x faster through optimization
  • uniq -c: 1.13x faster

• Comprehensive Benchmarking Infrastructure

  • Integrated CodSpeed for continuous performance monitoring
  • Added benchmarks for 15+ utilities including sort, ls, uniq, du, base64, and more
  • Performance regression detection in CI

• Safe Directory Traversal

  • Implemented secure traversal using openat/unlinkat for rm, du, chmod, and chgrp
  • Enhanced security through the nix crate instead of unsafe libc calls

• Enhanced GNU Compatibility

  • 532 passing tests (83.91% compatibility) with comprehensive test suite expansion
  • Improved error handling and behavior matching across utilities
  • Better Unicode and non-UTF8 path support

• Developer Experience Improvements

  • Moved documentation and completion generation out of binaries for faster builds
  • Enhanced CI with parallel benchmark compilation
  • Improved Makefile support for various build configurations

• Contributions: This release was made possible by the dedication of 36 contributors, including 25 newcomers.

GNU Test Suite Compatibility:

Note: Test result changes are primarily due to upgrading our reference from GNU coreutils 9.7 to 9.8, which introduced 16 new tests and updated existing test expectations.

Call to Action:

🌍 Help us translate - Contribute translations at Weblate
🚀 Sponsor us on GitHub to accelerate development: github.com/sponsors/uutils
🔗 Download the latest release: https://uutils.github.io

What's Changed

base64

• Input reuse on base64 by @ThePseudo in #8587
• Add benchmark for base64 by @ThePseudo in #8768

basenc

• basenc: implement --base58 encoding option by @AnarchistHoneybun in #8751
• base58: it wasn't working properly with long input by @sylvestre in #8818

chgrp

• safe traversal: adjust chmod & chgrp to use it by @sylvestre in #8632

chmod

• safe traversal: adjust chmod & chgrp to use it by @sylvestre in #8632

cksum

• cksum: improve performance (Closes: #8573) by @RenjiSann in #8643

cp

• cp: fix Ioctl casting for ppc64 (Closes: #8161) by @maxfilov in #8637
• Replace casting to Ioctl with a const from libc by @maxfilov in #8641
• cp: fix recursive cp fails on files with trailing backslashes by @Anonymous-AAA in #8747
• feat(cp): optimize directory copy by caching file checks and refactoring calls by @mattsu2020 in #8805
• cp: fix crash on -T/--no-preserve-target by @Alonely0 in #8864
• cp: show relative path of source file with -vr by @Ada-Armstrong in #8887
• cp: display symlink creation with --verbose by @Ada-Armstrong in #8899
• docs: add benchmarking guidelines for cp by @mattsu2020 in #8807

csplit

• csplit: create final empty file with --suppress-matched to match GNU (fixes #7286) by @naoNao89 in #8655

date

• date: use --reference=file to display the file modification time by @yuankunzhang in #8630
• date: consolidate date parsing logic by @yuankunzhang in #8629
• date: implement --resolution flag by @Banzobotic in #8686
• date: follow GNU pure-number -d semantics by @naoNao89 in #8907

df

• df: follow symlinks by @bakanovskii in #8763
• df: add tracing zero and rounding by @bakanovskii in #8685
• df: treat env var with zero block size same as invalid by @Ada-Armstrong in #8932
• tests(df): Fix test_total to avoid whitespace parsing issues by @naoNao89 in #8799

dirname

• Fix dirname handling of paths ending in /. by @naoNao89 in #8911

du

• Adjust rm & du to use safe traversal (openat, unlinkat, etc) by @sylvestre in #8517
• du: some small refactorings by @cakebaker in #8631
• du: add benchmark by @sylvestre in #8774
• du: ignore env zero blocksize by @Ada-Armstrong in #8961

expand

• expand: add a benchmark by @sylvestre in #8800
• expand: improve the performances - 1.80 faster than GNU by @sylvestre in #8803

expr

• expr: fix some multibyte issues by @sylvestre in #8606

fold

• fold: add a benchmark by @sylvestre in #8801
• fold: improve the performances - now same as GNU by @sylvestre in #8804

hashsum

• hashsum: fix help text to show actual utility name by @AnarchistHoneybun in #8650
• hashsum: remove the usage of the mut by @sylvestre in #8692
• Patch to add hashsum symlinks with tests by @tpowa in #8647
• hashsum: improve hashsum using 32KiB bufreader by @psvri in #8869

hostname

• hostname: fix -d flag by @bakanovskii in #8657
• Build {host,u}name on Windows by @oech3 in #8950

id

• id: added '-a' argument by @erammos in #8733

install

• tests/install/install-Z-selinux: fix selinux for install by @sylvestre in #8307

ln

• Fix ln -f handling when source and destination are the same entry by @mattsu2020 in #8838
• Allow to replace ln -fs and hardlink on Windows by default by @oech3 in #8810

ls

• ls: Lazily obtain FileType, eagerly obtain Metadata when Metadata is available by @kimono-koans in #8753
• ls: remove unnecessary getgrgid handling for Redox OS by @Connor-GH in #8779
• ls: add some benchmarks by @sylvestre in #8738
• ls: Implement -f flag to disable sorting and enable -a by @naoNao89 in #8824
• ls: fix zero block size handling to match GNU ls by @yrakcaz in #8928
• tests(ls): Add GNU-compat TIME_STYLE tests (#4627) by @naoNao89 in #8796

mv

• mv: support moving source to a symlink directory by @yuankunzhang in #8589
• mv: improve the verbose mode to make tests/mv/mv-special-1.sh pass by @sylvestre in #8521

nl

• nl: add a benchmark by @sylvestre in #8808
• nl: improve the performances by @sylvestre in #8811

nohup

• test(nohup): improve coverage for fd replacement and error paths by @naoNao89 in #8920

numfmt

• numfmt: add benchmark by @sylvestre in #8726

od

• od: fix options after filename issue by @Anonymous-AAA in #8624
• od: add support for -S by @sylvestre in #8849
• od: fix incomplete test & apply small refactorings to tests by @cakebaker in #8941

pinky

• pinky: fix time format by @bakanovskii in #8666
• pinky: add --lookup flag by @bakanovskii in #8669
• pinky: move unit test away from the integration tests by @cakebaker in #8671

ptx

• fix(ptx): Remove extra space before truncation marker and add a related test by @Misakait in #8784
• fix(ptx): Correct various output formatting bugs by @Misakait in #8787
• fix(ptx): Correct reference format for stdin by @Misakait in #8806
• fix(ptx): Align text wrapping behavior with GNU in traditional mode by @Misakait in #8820

realpath

• realpath: implement -E by @sylvestre in #8757

rm

• Adjust rm & du to use safe traversal (openat, unlinkat, etc) by @sylvestre in #8517
• rm: add the --progress option like with cp & mv by @sylvestre in #8567

seq

• seq: support non-utf8 for --separator & --terminator by @cakebaker in #8681
• clippy: move unexpected_cfgs to workspace lints & fix warnings in seq by @cakebaker in #8859

sort

• sort: add benchmark by @sylvestre in #8735
• sort: adapt error message to pass sort-files0-from by @cakebaker in #8748
• sort: also declare sort_locale_bench as benchmark by @sylvestre in #8780
• sort: add a bench for long line by @sylvestre in #8886
• sort: fix newline handling across large and/or multiple files by @jacob-greenfield in #8746
• Reduce variance in sort benchmarks by reusing temp file by @naoNao89 in #8939
• feat(sort): auto-tune buffer sizing from available memory by @mattsu2020 in #8959
• Improve sort buffer sizing heuristics and honor explicit --buffer-size (#8833)

stat

• stat: fix mount point handling for non-UTF8 paths by @dekuu5 in #8538
• stat: fix %N symlink to be single quote instead of double quote by @dekuu5 in #8791
• Include subsecond precision in stat atime/ctime by @gaul in #8903

stdbuf

• stdbuf: fix a publish issue by @sylvestre in #8566

stty

• move stty to UNIX_PROGS by @oech3 in #8925

tail

• Add tests/tail/overlay-headers to the intermittent list by @sylvestre in #8736

tee

• tests(tee): Add GNU-compat write-error and broken-pipe tests (#4627) by @naoNao89 in #8797

timeout

• tests/timeout: remove unnecessary stderr check for test_kill_subprocess by @elliotwesoff in #8845

touch

• touch: adapt expected error message in test by @cakebaker in #8599
• touch: simplify some tests by using new_ucmd!() by @cakebaker in #8602
• Remove additional touch when feat_common_core is required by @zdiff in #8598
• GNUmakefile: move du & touch to PROGS by @cakebaker in #8926

tsort

• tsort: write a benchmark by @sylvestre in #8712
• Remove extra lookups and memory allocations from tsort graph construction by @Nekrolm in #8694

unexpand

• unexpand: add a benchmark by @sylvestre in #8813
• unexpand: improve performances (1.50 faster than GNU) by @sylvestre in #8815
• unexpand: remove unnecessary condition by @cakebaker in #8822
• unexpand: add support for non-utf8 filenames by @cakebaker in #8836

uniq

• uniq: add benchmarks by @sylvestre in #8762
• Refactor uniq field skipping to avoid allocations by @mattsu2020 in #8703
• uniq: reduce the size of the benchmarks by @sylvestre in #8764

uname

• Build {host,u}name on Windows by @oech3 in #8950

wc

• wc: adapt error message to pass wc-files0-from by @cakebaker in #8749

who

• who/pinky: fix systemd local host by @bakanovskii in #8672

uucore

• uucore/safe_traversal: remove unnecessary vars by @cakebaker in #8627
• safe traversal: use the nix crate instead of unsafe calls to libc by @sylvestre in #8626
• uucore/checksum: use ChecksumError variants by @cakebaker in #8636
• uucore/selinux: add support for translations by @sylvestre in #8628
• uucore/checksum: improve API of create_sha3() by @cakebaker in #8645
• uucore: parse_time: avoid expensive with_scale(9) for huge inputs by @loskutov in #8740
• uucore: fix clippy warnings by @bakanovskii in #8769
• uucore/fsext: recognize magic number for cgroup2fs by @cakebaker in #8778
• uucore: handle situation using numberic username by @zhw2101024 in #7788
• uucore/parse_time: return 1ns for small numbers by @cakebaker in #8821
• uucore: ringbuffer pre-allocates memory based on size given by @asder8215 in #8895
• refactor(uucore): integrate procfs for Linux memory parsing by @mattsu2020 in #8951

Localization & Internationalization

• l10n: replace the hash table by a long match per file to faciliate th… by @sylvestre in #8734

Performance & Benchmarking

• evaluate codspeed for perfs by @sylvestre in #8680
• github: improve the codspeed configuration by @sylvestre in #8689
• fix the divan usage for codspeed by @sylvestre in #8710
• bench: call the function directly to improve the profiling + move the common functions in uucore by @sylvestre in #8716
• Codspeed: reduce the number of tests and increase some values for long tests by @sylvestre in #8758
• bench: remove some duplication by @sylvestre in #8781
• fix some benchmark (some of them were failing) by @sylvestre in #8773
• More bench by @sylvestre in #8870
• Document the bench process by @sylvestre in #8809

Security

• Document the security process by @sylvestre in #8633
• add a github check for programs not using traversal by @sylvestre in #8698

Code Quality & Cleanup

• Minor fix by @dev-4hmad in #8596
• systemd_logind: replace i8 with libc::c_char by @Ecordonnier in #8595
• clippy: add missing [lints] workspace = true to uucore/Cargo.toml by @cakebaker in #8771
• clippy: re-enable semicolon_if_nothing_returned by @cakebaker in #8775
• clippy: re-enable three more lints by @cakebaker in #8777
• clippy: move use_self lint to workspace lints by @cakebaker in #8839
• clippy: re-enable manual_let_else lint by @cakebaker in #8894
• Cargo.toml: remove lints.clippy section by @cakebaker in #8897
• chore: huge clippy pass ahead of 1.92.0 release by @Alonely0 in #8900
• Clippy fix by @devnexen in #8991
• codegen-units = 1 at release-{fast,small} profile by @oech3 in #8794
• Cargo.toml: remove two features by @cakebaker in #8954
• Cargo.toml: cleanup release-small profile by inherits by @oech3 in #8969
• deny.toml: remove nix from skip list by @cakebaker in #8967
• fix procfs dependencies by @mattsu2020 in #8968

CI & Build

• make install: ressource files should have 644 as permissions. Closes: #8638 by @sylvestre in #8640
• ci: fix broken CICD workflow by @cakebaker in #8831
• ci: add apt-get update to code-quality workflow by @cakebaker in #8865
• ci: test platform-specific utils separately, too by @cakebaker in #8846
• github: compile & run benchmarks in parallel by @sylvestre in #8891
• upgrade to GNU coreutils 9.8 as ref by @sylvestre in #8720
• build-gnu.sh: improve setup instructions by @cakebaker in #8731
• fuzz: enable debug symbols in release builds by @naoNao89 in #8935
• 2 makefile fixes by @Ecordonnier in #8782
• makefile: fix build with overriden UTILS by @Ecordonnier in #8702
• GNUmakefile: Simplify installing mans and completions by @oech3 in #8978
• build: Move unlink and arch to PROGS by @oech3 in #8958

Documentation

• docs: minor grammar fix in packaging.md by @dev-4hmad in #8620
• Fix the doc generation and remove uuhelp_parser by @sylvestre in #8610
• Add missing manpages by @oech3 in #8721
• Add missing completions by @oech3 in #8723
• Document how to generate prefixed completions by @oech3 in #8817
• doc: add -h & -V to extensions by @cakebaker in #8888
• doc: How to build for SELinux by cargo by @oech3 in #8964
• uudoc: move manpage and completions generation out of binary by @aaron-ang in #8513

Platform Support

• fix: Add cross-platform stubs and gate SELinux to Linux-only by @naoNao89 in #8795
• Drop redundant || exit 1 by @oech3 in #8881

Version Management

• --version should just print the command name, not the path by @sylvestre in #8921
• prepare version 0.3.0 by @sylvestre in #8890

Dependency Updates

• fix(deps): update rust crate tempfile to v3.22.0 by @renovate[bot] in #8590
• chore(deps): update rust crate zip to v5.0.1 by @renovate[bot] in #8597
• chore(deps): update rust crate zip to v5.1.0 by @renovate[bot] in #8609
• Bump rustix from 1.0.7 to 1.1.2 by @cakebaker in #8613
• Bump fts-sys and selinux-sys by @cakebaker in #8612
• chore(deps): update rust crate zip to v5.1.1 by @renovate[bot] in #8616
• chore(deps): update actions/checkout action to v5 by @renovate[bot] in #8623
• chore(deps): update rust crate serde to v1.0.221 by @renovate[bot] in #8635
• chore(deps): update rust crate serde to v1.0.222 by @renovate[bot] in #8639
• chore(deps): update rust crate serde to v1.0.223 by @renovate[bot] in #8642
• chore(deps): update rust crate serde to v1.0.224 by @renovate[bot] in #8646
• chore(deps): update rust crate serde to v1.0.225 by @renovate[bot] in #8649
• chore(deps): update rust crate clap_complete to v4.5.58 by @renovate[bot] in #8648
• chore(deps): update rust crate time to v0.3.44 by @renovate[bot] in #8668
• chore(deps): update rust crate clap to v4.5.48 by @renovate[bot] in #8675
• chore(deps): update rust crate xattr to v1.6.1 by @renovate[bot] in #8688
• chore(deps): update rust crate serde to v1.0.226 by @renovate[bot] in #8687
• chore(deps): update vmactions/freebsd-vm action to v1.2.4 by @renovate[bot] in #8700
• fix(deps): update rust crate tempfile to v3.23.0 by @renovate[bot] in #8714
• chore(deps): update rust crate memchr to v2.7.6 by @renovate[bot] in #8851
• chore(deps): update rust crate quote to v1.0.41 by @renovate[bot] in #8852
• chore(deps): update rust crate serde to v1.0.228 by @renovate[bot] in #8854
• chore(deps): update rust crate regex to v1.11.3 by @renovate[bot] in #8853
• chore(deps): update rust crate unicode-width to v0.2.2 by @renovate[bot] in #8856
• chore(deps): update rust crate windows-sys to v0.61.2 by @renovate[bot] in #8857
• chore(deps): update rust crate thiserror to v2.0.17 by @renovate[bot] in #8855
• chore(deps): update rust crate zip to v6 by @renovate[bot] in #8860
• chore(deps): update rust crate ctor to 0.6.0 by @renovate[bot] in #8850
• Bump half & zerocopy by @cakebaker in #8866
• chore(deps): update rust crate serde to v1.0.228 by @renovate[bot] in #8875
• chore(deps): update rust crate thiserror to v2.0.17 by @renovate[bot] in #8876
• chore(deps): update rust crate quote to v1.0.41 by @renovate[bot] in #8874
• chore(deps): update rust crate memchr to v2.7.6 by @renovate[bot] in #8873
• chore(deps): update rust crate unicode-width to v0.2.2 by @renovate[bot] in #8877
• chore(deps): update rust crate regex to v1.12.1 by @renovate[bot] in #8879
• chore(deps): update rust crate windows-sys to v0.61.2 by @renovate[bot] in #8878
• Bump nu-ansi-term from 0.50.1 to 0.50.3 by @cakebaker in #8885
• Bump half & zerocopy by @cakebaker in #8884
• chore(deps): update rust crate regex to v1.12.2 by @renovate[bot] in #8905
• chore(deps): update rust crate clap_complete to v4.5.59 by @renovate[bot] in #8902
• chore(deps): update rust crate clap_mangen to v0.2.30 by @renovate[bot] in #8904
• chore(deps): update rust crate clap to v4.5.49 by @renovate[bot] in #8901
• chore(deps): update rust crate half to v2.7.1 by @renovate[bot] in #8909
• chore(deps): update rust crate lscolors to 0.21.0 by @renovate[bot] in #8957
• chore(deps): update rust crate bigdecimal to v0.4.9 by @renovate[bot] in #8956
• chore(deps): update rust crate divan to v4 by @renovate[bot] in #8858
• chore(deps): update rust crate clap to v4.5.50 by @renovate[bot] in #8965
• chore(deps): update rust crate dns-lookup to v3.0.1 - autoclosed by @renovate[bot] in #8966
• chore(deps): update rust crate indicatif to v0.18.1 by @renovate[bot] in #8971
• chore(deps): update rust crate memmap2 to v0.9.9 by @renovate[bot] in #8972
• chore(deps): update rust crate self_cell to v1.2.1 by @renovate[bot] in #8973
• chore(deps): update rust crate clap_mangen to v0.2.31 by @renovate[bot] in #8980
• chore(deps): update rust crate proc-macro2 to v1.0.102 by @renovate[bot] in #8981
• chore(deps): update rust crate proc-macro2 to v1.0.103 by @renovate[bot] in #8989

New Contributors

• @dev-4hmad made their first contribution in #8596
• @zdiff made their first contribution in #8598
• @ThePseudo made their first contribution in #8587
• @maxfilov made their first contribution in #8637
• @naoNao89 made their first contribution in #8655
• @bakanovskii made their first contribution in #8666
• @AnarchistHoneybun made their first contribution in #8650
• @Banzobotic made their first contribution in #8686
• @tpowa made their first contribution in #8647
• @dekuu5 made their first contribution in #8538
• @erammos made their first contribution in #8733
• @loskutov made their first contribution in #8740
• @Nekrolm made their first contribution in #8694
• @mattsu2020 made their first contribution in #8703
• @Connor-GH made their first contribution in #8779
• @Misakait made their first contribution in #8784
• @elliotwesoff made their first contribution in #8845
• @Alonely0 made their first contribution in #8864
• @Ada-Armstrong made their first contribution in #8887
• @jacob-greenfield made their first contribution in #8746
• @asder8215 made their first contribution in #8895
• @gaul made their first contribution in #8903
• @yrakcaz made their first contribution in #8928
• @psvri made their first contribution in #8869

Full Changelog: 0.2.2...0.3.0