juicedata/juicefs v0.17.0

on GitHub

JuiceFS v0.17 arrived one month after 0.16, with 80+ commits from 9 contributors (@SandyXSD, @davies, @xiaogaozi, @yuhr123, @Suave @tangyoupeng @201341 @zwwhdls @allwefantasy), thanks to them!

This release improved the performance when JuiceFS is used for temporary data by using an in-memory meta engine (memkv) and delayed uploading. For example, we can use JuiceFS as the shuffle and spill disk without worrying about running out of space.

Linux Test Project was used to verify the compatibility of JuiceFS, please checkout the current results here.

This release introduced metadata cache for Java SDK and S3 gateway (similar to metadata cache in kernel), which could be turned to improve performance significantly.

New

• Added a option to delay upload in writeback mode (#736 #743), which is useful for temporary data.
• Added an in-memory meta engine (memkv://) for temporary data (#751 #779).
• Support CREATE and REPLACE flag in setxattr (#770).
• Added metrics for in-memory cache (#776).
• Support rename flags (NoReplace and Exchange) (#787).
• New colorful result for bench command (#810).
• Added entry and attributes cache for Java SDK and S3 gateway (#835).

Changed

• Default logging directory in macOS was changed to user's home directory (#744).
• Limit the number of retry for listing to 3 to avoid infinite loop (#745).
• Show total size of valid objects in gc command (#746).
• Disable SHA256 checksum for S3 and other compatible object store to reduce CPU usage (#754).
• Hide operations on internal files from access log (#766).
• Require Go 1.15 to build JuiceFS and build the release with Go 1.16 (#771).
• Inherit gid of parent when SGID is set (#772).
• Keep SGID when file is non-group-executable (#773).
• Allow remove broken dir/file (#784).
• Retry transactions for TxnLockNotFound from TiKV (#789).
• Cleanup current session during umount (#796).
• Reduce memory allocation for OBS and Webdav backend (#800).
• Support escaped access key for KS3 (#830).
• Support lookup . and .. (#842).
• No warning if compaction fail with missing objects(#844).
• Increase available inodes based on current usage (#851).
• Allow update access key and secret key with default compress algorithm (#855).

Bugfix

• Fixed a leak in SQL engine (#728).
• Fixed a bug that may crash client (#729).
• Fixed valid bytes of progress bar for gc command (#746).
• Fixed warmup with a long list of files (#752).
• Fixed supporting of secured Redis connection (regression in v0.16) (#758).
• Fixed data corruption in SQL and TiKV engine when some slices are skipped during compaction (#759).
• Fixed metrics for read bytes in Hadoop SDK (#761).
• Fixed multipart upload in S3 gateway (#765).
• Fixed POSIX locks on interweaved regions (#769).
• Fixed latency metrics for libRADOS (#793).
• Fixed concat in Java SDK and multipart upload (#817).
• Fixed nlink of parent when rename directories (#839).
• Fixed transaction for read-only mount (#844).