32-bit glibc Build Fixes and FUSE Driver Cleanup

32-bit native build fixes (#354)

This release fixes a set of issues that showed up in native 32-bit builds, most notably on openSUSE Tumbleweed with glibc. The most important one was related to sparse file tests that create files larger than 4 GiB: glibc requires _FILE_OFFSET_BITS=64 for 64-bit file operations in that environment, and without it those tests would fail. This was not just a test bug, all the binaries simply could not deal with files larger than 4 GiB in this scenario.

However, this was only an issue for glibc-based 32-bit builds. It did not affect musl-libc-based builds, which is why it went unnoticed for some time. This did not affect the statically linked release binaries, which are using musl-libc.

A few additional 32-bit related issues were also fixed: the formatting code for times and ratios relied too heavily on floating-point arithmetic and turned out not to be deterministic across platforms and compilers. That code has now been rewritten to use integer arithmetic instead, avoiding platform-specific behavior. A bug in the test code itself, which surfaced only with GCC, has been fixed as well.

FUSE driver refactoring

The FUSE drivers (dwarfs, dwarfs2) have gone through a substantial internal cleanup and refactoring pass in this release. This fixes a number of subtle startup and option-handling issues and makes the behavior of the different driver variants much more consistent.

One visible improvement is error reporting for invalid options. For example, if -o image_size=1234 was used instead of the correct -o imagesize=1234, older versions would end up reporting a misleading filesystem loading failure rather than flagging the unknown option directly. That kind of behavior has now been fixed.

Internally, this also removes a great deal of preprocessor-heavy startup logic that had diverged over time across the various FUSE implementations and modes (high-level, low-level, Windows, FUSE v2, and FUSE v3). The resulting code is much easier to follow, more consistent across platforms, and better covered by tests.

Bug fixes

The image size was not passed correctly to one instance of the filesystem parser, which caused errors when loading a DwarFS image embedded inside a larger file (for example, a multi-layer file). Thanks to Ruan Formigoni for the pull request fixing this issue.
The performance monitor timer for op_lseek in the FUSE driver was not initialized correctly. This could lead to segmentation faults or bus errors due to an uninitialized index into a std::deque. The issue has been fixed, and an additional check has been added to catch similar errors in the future.
Native 32-bit glibc builds could fail in tests involving sparse files larger than 4 GiB because _FILE_OFFSET_BITS=64 was required for 64-bit file operations. Additional non-deterministic test failures related to time and ratio formatting were also fixed by rewriting that code to use integer arithmetic instead of floating-point arithmetic. A separate test bug that surfaced only with GCC was fixed as well. Addresses #354.
The FUSE drivers (dwarfs, dwarfs2) were refactored to eliminate a number of subtle startup and option-handling issues and to make behavior much more consistent across the various supported FUSE variants.
The manual pages shown with --man for all tools unintentionally included the license header from XML comments at the top of the source files. The renderer now ignores those comments.

Build

After benchmarking the latest mimalloc allocator, it turns out to perform mostly on par with jemalloc. It is still less configurable, but it is clearly usable when that extra configurability is not needed.
The small universal release binaries are now built with mimalloc instead of jemalloc, reducing their size by about 10%.
Older Clang versions, such as those shipped with Ubuntu 22.04, are no longer supported because they cannot use libstdc++'s std::expected implementation. DwarFS can still be built on Ubuntu 22.04 with GCC.

Test

A few particularly slow tests were identified through profiling and have been reworked to run faster while still covering most of the same code paths.

New Contributors

@ruanformigoni made their first contribution in #355

Full Changelog: v0.15.1...v0.15.2

SHA-256 Checksums

f81290d5c4890a274a60a82b801e62110975ca466799e4408ad046bc70695aef  dwarfs-0.15.2-Linux-aarch64.tar.xz
2466659a4e783d1bc794e17215dcdf093fae8e9493ffaa03091a85c8c11369f2  dwarfs-0.15.2-Linux-arm.tar.xz
fd6ed59b7805539c6fc6e6e508344d10479f6587abe1040a1e86997f19b52fa0  dwarfs-0.15.2-Linux-i386.tar.xz
62f5f2bac65232184cb14e7bb7921a38b34440dcd688080eebdcdc512915d3ba  dwarfs-0.15.2-Linux-loongarch64.tar.xz
08e556adeb248a20aec04bf70886146f5e56bd0efc0bbe225aae69d20b25d33e  dwarfs-0.15.2-Linux-ppc64le.tar.xz
1668eca05412c2bd848ad4aa606a61fb2bf48f3cf92aac556149778a713885f2  dwarfs-0.15.2-Linux-ppc64.tar.xz
3110c8f630a4ae65fe16e4e3280f74d34b2c7a8224630e03361ed799819eeb9f  dwarfs-0.15.2-Linux-riscv64.tar.xz
5879d6d1e406a5ddf4dc68d8179540bd797af5d58abe7def9e62c6841221eb95  dwarfs-0.15.2-Linux-s390x.tar.xz
ad9ef4edf2d124b5c8d4540b2298a9b520e15841c68b921192d9bcb83eb69455  dwarfs-0.15.2-Linux-x86_64.tar.xz
6b7edcb2121347e273753d949f72913f006ec12477248384b24c48989ec34995  dwarfs-0.15.2.tar.xz
c34c05e7f65edcfd5f21bc3fed2de90d3f4b8c5b311d3b5ae302ae3ec6fc7161  dwarfs-0.15.2-Windows-AMD64.7z
46d690ecf546539d3229df9e3ad2412a806e4dee95e22c9ea843d934003f21bb  dwarfs-fuse-extract-0.15.2-Linux-aarch64
3d49d59416bcb989504b70bc63ebcbbb5be5c9a049bd432cefeb4620aa0fc3ae  dwarfs-fuse-extract-0.15.2-Linux-aarch64.upx
d6b4f4fc498e48031f7e7bb5ed3297a1272a68465f1803ad599ee486b0d49799  dwarfs-fuse-extract-0.15.2-Linux-arm
c5214353ad9de53a7fbff6aa5adfa65a9b830b694be839d5de1dbec9ff6a12f0  dwarfs-fuse-extract-0.15.2-Linux-arm.upx
5056562a6e12bcb20e7e07aeff0fa31113bae52f6feefd1a79f87d412c8bcd6d  dwarfs-fuse-extract-0.15.2-Linux-i386
92ac91f59581663985250cc0bcad02742c5cf1a5b009925212f97c1a424f69c3  dwarfs-fuse-extract-0.15.2-Linux-i386.upx
ede37a302b6e038f25dc4a35f25d8c9378fb2ae35600ec445a1eec0bf8b9d64c  dwarfs-fuse-extract-0.15.2-Linux-loongarch64
eec1b96f2d19c04aeda2099914e0b905cec6dec98be439f1bfc4f8f31c447554  dwarfs-fuse-extract-0.15.2-Linux-ppc64
983ee30f114bbafdeefc7ca8a79eecffa33c6314b9c516e7183424ec1d75802c  dwarfs-fuse-extract-0.15.2-Linux-ppc64le
fba9d984f3258f5c35e441b2eb468e25baa9f339e6100013c5a2fbb1c6bfcc1e  dwarfs-fuse-extract-0.15.2-Linux-riscv64
a3210163095b118407f6b8bd174d60bdb92a9d3fa70b9cdcb761d39570bd2bbb  dwarfs-fuse-extract-0.15.2-Linux-riscv64.upx
28f9ca8b2c9ccfc3c7e5949ed89b33a24aade539b3e0de40d5d364e0d06b0065  dwarfs-fuse-extract-0.15.2-Linux-s390x
85084aae43406fa8892df1cc24bd6df96948d99e8a35ca4c2873518fef0fa41c  dwarfs-fuse-extract-0.15.2-Linux-x86_64
40d027c79c7be4a5f7701433477f832390c3aaab437ca3bb2e3f0aa0a007d557  dwarfs-fuse-extract-0.15.2-Linux-x86_64.upx
b9c25399e1e4672601c0afbcb44e522af71698ea41691384555abe09776c113e  dwarfs-universal-0.15.2-Linux-aarch64
432a460080765f52b2b6e51ef99ca87bc4ba1cdf8fa9257155cd89d1b20edbc8  dwarfs-universal-0.15.2-Linux-aarch64.upx
114496a88f27af058481d09e4c75a5079ff9567317745fd50ca4ae79eec30070  dwarfs-universal-0.15.2-Linux-arm
23703b2ce7de3a3ed52f1441759f91a10ab654243678a050e98e5fda0d821c05  dwarfs-universal-0.15.2-Linux-arm.upx
35771934f6dc34038624cf99179bcfb4fe707c533e294d4a75cebdc56785cf08  dwarfs-universal-0.15.2-Linux-i386
24d6ae52dc7166d667cabb47b77233f3a2e87848fb7ffc71a1d230aeb54dd5f7  dwarfs-universal-0.15.2-Linux-i386.upx
a3d97d40a2cc1baf70efe5e5509f9537826b13ee27bdf3bf95a2914361d31181  dwarfs-universal-0.15.2-Linux-loongarch64
547650e46dde48abdd5824115e3c39cb66d551e4647383716591ea5bbcfb7f2e  dwarfs-universal-0.15.2-Linux-ppc64
9712e02402f2a2094a565d39ec2bad2a3a43dbc042a18c93035726de4b133c24  dwarfs-universal-0.15.2-Linux-ppc64le
24f53482d4992797b178e9d63156519569168d2ad469c998451670c44362c418  dwarfs-universal-0.15.2-Linux-riscv64
912b7c0205df9c3a3af61e9b5816fd7b83219259db3ecbe9d0761be97335d0c1  dwarfs-universal-0.15.2-Linux-riscv64.upx
13719648b094adf2c86df6dcc1c34f6b414d01f2032ab1c035bcc95e97f7d135  dwarfs-universal-0.15.2-Linux-s390x
7e4d7ed1b4c8d3f1db01d6914e5b8bcba1d1eb69dfaafe06d32214797fb0dd55  dwarfs-universal-0.15.2-Linux-x86_64
3f073cf225f975415da316aacee1825497a981ff7dad654bf50adb9b2592f7bb  dwarfs-universal-0.15.2-Linux-x86_64.upx
af985a17f080dd708f4fe8c6993adfa0a9f2545f96322e86dd4464ec2bcc0838  dwarfs-universal-0.15.2-Windows-AMD64.exe
feb6e82cd37eba74540c20ab4b73f82c5698f37bada15c7869ab976e04792c46  dwarfs-universal-small-0.15.2-Linux-aarch64
622c917795af04b045d070846ef8492f1c033f5722aa2c02549e218bcaaeace3  dwarfs-universal-small-0.15.2-Linux-aarch64.upx
759c66628b384769e9a3bcd5f98feadec3faa5334910e865c57551a44258f817  dwarfs-universal-small-0.15.2-Linux-arm
2b2fb1d586d79eff8024f69ffc805a341323358ef96ea7cbaf06b41fb76e9628  dwarfs-universal-small-0.15.2-Linux-arm.upx
041cd1200eeac5cf6bdd0c8dc922d119390959a4881b70eddf82d7ac79ae8d89  dwarfs-universal-small-0.15.2-Linux-i386
1dee9fc1846b902f544d8fcfaf7299b0381fa5d8345c2e9eb5d581ce73350b86  dwarfs-universal-small-0.15.2-Linux-i386.upx
c41cc39c690bba884077f995233eeeef968e6de20b2814fc4c1836281892ce70  dwarfs-universal-small-0.15.2-Linux-loongarch64
9f45a53288ac79192454aa5c96d90c92d47206a6162516b5d8f7efa6b85128a4  dwarfs-universal-small-0.15.2-Linux-ppc64
8a6d16cc922ba48b17c78964388cd04f5fa39593340a1721453ed3ebc766862c  dwarfs-universal-small-0.15.2-Linux-ppc64le
5f05442e1426ce962a0f319f0e27167943024758f5dd508053683a8cefa261a7  dwarfs-universal-small-0.15.2-Linux-riscv64
b84b860adee190cadd0d54a643d9eb787366320371d67944e3e0aff035a4acb5  dwarfs-universal-small-0.15.2-Linux-riscv64.upx
49cc48c441feb0a96b1685a6919ae361a8ae27e060e7a0ff1d92b8611091edaa  dwarfs-universal-small-0.15.2-Linux-s390x
85f0a2a95c8438a06e1177936ae1e4d9e45cec7876824037776b7866cb19973c  dwarfs-universal-small-0.15.2-Linux-x86_64
ff16901aa36b692a98bacfda58c68c5868ff4dd72babe61b33571165730cf477  dwarfs-universal-small-0.15.2-Linux-x86_64.upx

mhx/dwarfs v0.15.2 dwarfs-0.15.2 on GitHub