JuiceFS v1.0.0-rc1 is the first release candidate for v1.0. It has 184 commits from 17 contributors, thanks to @davies @zhoucheng361 @SandyXSD @zhijian-pro @sanwan @xiaogaozi @tangyoupeng @solracsf @showjason @rayw000 @AIXjing @helix-loop @Suave @zhouaoe @chnliyong @yuhr123 @liufuyang !

Highlights

• Dumping metadata from Redis has been improved, massively reducing the memory required to below 1/20 of that before. It also relieves the memory spike of a client when doing metadata backup. Dumping metadata from SQL and TiKV within a single transaction to ensure consistency.
• Loading metadata to all engines has been improved as well. Instead of loading the whole dumped file in one step, JuiceFS will now read it in a stream, and simultaneously import metadata to the engine. This saves a lot of memory when the dumped file is huge.
• Improved stability for SQL engine under heavy workload.
• Added a new command juicefs objbench that can be used to run basic function tests and benchmarks on object storage, making sure it works as expected.

New

• Supports using SQL databases, etcd as data storage (#2003, #2009)
• Supports finding all paths of an inode with juicefs info command (#2058, #2161, #2193)
• Supports using Pyroscope to record JuiceFS profiling (#1952)
• Added progress bar for juicefs rmr and juicefs warmup commands (#2197)
• Added a new command juicefs objbench to run basic benchmarks on object storage (#2055, #2091)
• Added a new command juicefs version to print version, as an alternative to --version (#2229)

Changed

• cmd: check the range of parameters (#2195)
• cmd: eliminate panic which is triggered by missing argument (#2183)
• cmd/mount: warn about the behavior of mounting the same directory multiple times (#2141)
• cmd/warmup: support warmup from inside of a container (#2056)
• meta: add delayed slice only when chunkid > 0 (#2231)
• meta: speed up and reduce memory for loading metadata (#2142, #2148)
• meta: add a pessimistic lock to reduce conflicted transactions in the database (#2111)
• meta: limit the number of scanned files in cleanup (#2157)
• meta: limit number of files when cleanup trash (#2061)
• meta: limit the number of coroutines to delete file data (#2042)
• meta: log last error if a transaction has been ever restarted (#2172)
• meta: log session info when cleaning up a stale one (#2045)
• meta: skip updating mtime/ctime of the parent if it's updated recently (#1960)
• meta/redis: check config 'maxmemory-policy' (#2059)
• meta/redis: Speedup dump for Redis and reduce memory usage (#2156)
• meta/tkv: Speedup dump for kv storage (#2140)
• meta/tkv: dump metadata using snapshot (#1961)
• meta/tkv: use scanRange to get delayed slices (#2057)
• meta/sql: dump metadata in a single transaction (#2131)
• chunk/store: keep cache after uploading staging blocks (#2168)
• object: reload the configuration for data storage (#1995)
• object/sftp: load default private keys for sftp (#2014)
• object/oss: add user agent for oss #1992 (#1993)
• vfs: hide .control from readdir (#1998)
• gateway: clean up expired temporary files (#2082)
• SDK: package amd64 and arm64 libjfs (#2198)
• SDK: don't reuse fd in Java SDK (#2122)
• Dependency: upgrade coredns for CVE-2019-19794 (#2190)
• Dependency: upgrade azblob sdk (#1962)
• meta: keep valid utf8 in dumped JSON (#1973)
• SDK: mvn shade some dependency to avoid class conflict (#2035)
• meta: truncate trash entry name if it's too long (#2049)
• meta/sql: use repeatable-read for transaction (#2128)

Bugfix

• cmd: fix not showing arguments for commands without META-URL (#2158)
• cmd/sync: fix sync lost file (#2106)
• cmd/warmup: fix warmup on read-only mount point (#2108)
• meta: skip updating sliceRef if id is 0 (#2096)
• meta: fix update xattr with the same value (#2078)
• meta/redis: handle lua result from Redis v7.0+ (#2221)
• meta/sql: fix dump with more than 10000 files (#2134)
• meta/sql: one transaction in SQLite at a time (#2126)
• meta/sql: fix rename with Postgres with repeatable read (#2109)
• meta/sql: fix deadlock in PG (#2104)
• meta/sql: ignore error about duplicated index (#2087)
• meta/sql: read database inside transaction (#2073, #2086)
• meta/sql: retry transaction on duplicated entry and concurrent update (#2077)
• meta/sql: fix the deadlock in rename (#2067)
• meta/sql: retry transaction for duplicate key in PG (#2071)
• meta/sql: fix update query in SQL transaction (#2024)
• meta/tkv: fix value of delSliceKey (#2054)
• meta/tkv: upgrade TiKV client to 2.0.1 to fix nil connection (#2050)
• chunk/store: fix stats of cached space in writeback mode (#2227)
• object: delete should be idempotent (#2034)
• object/file: Head of file should return File (#2133)
• object/s3: check prefix and marker with returned keys from S3 (#2040)
• object/prefix: fix with prefix returning nil error for unsupported ops (#2021)
• object/sftp: fix auth of sftp with multiple keys (#2186)
• object/sftp: fix prefix of sftp, support ssh-agent (#1954)
• vfs/backup: skip cleanup if list failed (#2044)
• SDK: handle atomic rename exception (#2192)