This is the third major release of JuiceFS, and is an LTS version as well that will be maintained for 24 months.
Starting from v1.1 it has 464 commits from 44 contributors, thanks to @zhijian-pro @davies @zhoucheng361 @SandyXSD @jiefenghuang @polyrabbit @CaitinChen @yuhr123 @Hexilee @tangyoupeng @YunhuiChen @eryugey @timfeirg @Suave @solracsf @kyungwan-nam @xiaogaozi @1ambd4 @stefanb @XDTD @occupyhabit @menwenjun @li1213987842 @testwill @fengleng @deferdeter @camcui @uvletter @T-TRz879 @yuchen-sun @TechQuery @reAsOn2010 @mtnk2005 @kanemathers @JoJossd @sjp00556 @CodingPoeta @abserari @AChangFeng @feeyman @l0wl3vel @chnliyong @JoyLiuc @prOOrc!
Highlights
- Supports graceful restart for mount process (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555, #4792, #4823, etc.)
- Supports POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
- Supports Identity and Access Management (IAM) and events notify for gateway (#4030, #4593, #4622, #4845, #4891)
New
- Added a new command
compact
to trigger compaction on a specified path (#4337) - Supports using NFS as object storage (#3823, #4101, #4105, #4111, #4147, #4163, #4718)
- Supports using Dragonfly as object storage (#4057)
- Supports using Bunny as object storage (#4719)
- cmd: add a new option
--log-id
to append user-specified ID in log (#4040) - cmd/mount: supports restarting mount process gracefully (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555, #4792, #4823, etc.)
- cmd/format: add a new option
--enable-acl
to enable POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640) - cmd/mount: add a new option
--skip-dir-mtime
to eliminate burst mtime updates of a directory (#4189) - cmd/mount: add a new option
--upload-hours
to limit the period for background uploads (#4250) - cmd/mount: add a new option
--cache-expire
to set expired time for cached blocks (#4255, #4365, #4434) - cmd/mount: add a new option
--custom-labels
to set labels for metrics (#4312, #4612) - cmd/mount: add a new environment variable
JFS_MOUNT_TIMEOUT
to adjust timeout for checking mount point ready (#4219) - cmd/mount: add a new option
max-stage-write
to enable concurrent write to both stage files and object storage in writeback mode (#4743) - cmd/gateway: supports IAM and events notify (#4030, #4593, #4622, #4845, #4891)
- cmd/gateway&webdav: add a new option
--background
to support daemon running (#3993, #4631, #4708, #4915) - cmd/gateway: add a new option
--domain
to support virtual-host-style requests (#4462) - cmd/warmup: add a new option
--evict
to manually evict cached blocks (#4370) - cmd/warmup: and a new option
--check
to check the ratio of already warmed blocks (#4370) - cmd/dump: add a new option
--fast=false
to reduce memory usage with a slower dump process (#4253) - cmd/dump: add a new option
--skip-trash
to skip the trash directory when backing up metadata (#4479, #4563) - cmd/dump: add a new option
--threads
to adjust speed for dump (#4581) - cmd/sync: add a new option
--inpace
to put file in-place, eliminating the rename operation (#4104) - cmd/sync: supports two stars
**
and three stars***
to match all files recursively (#4422, #4448, #4460, #4468, #4474) - cmd/sync: add a new option
--match-full-path
to match the filter against full path (#4492) - cmd/sync: add new options
max-size
,min-size
,max-age
andmin-age
to filter files based on size and age (#4912, 7491e33, #4913) - cmd/sync: add p8s metrics to better monitor the sync progress (#4119)
- meta: supports Open File Description (OFD) lock (#4083)
- chunk: disk cache switch to consistent hash and add bad disk detection (#4543, #4749, #4780, #4784, #4871, #4848)
- object: add query string to disable md5 and crc verification when s3 compatible storage does not support them (#4606)
- object: add a new environment variable
JFS_S3_VHOST_STYLE=1
to enable vhost style (#4697) - hadoop: add
push-labels
option to set custom labels for metrics (#4312)
Changed
- cmd: support human friendly values for options (#4246, #4558, #4567, #4592, #4641, #4671)
- cmd/mount: only use
writeback_cache
to enable fuse writeback (#4632) - cmd/gateway: judge loop symbolic links rather than hard-code deep limit (#4045)
- cmd/gateway: create MinioMetaBucket when it doesn't exist (#4383)
- cmd/gateway: create
.minio.sys
directory even if multi-buckets are not enabled (#4367) - cmd/dump: use a temporary file during dumping (#4767)
- cmd/sync: limit the maximum memory size for multiple uploads (#4135)
- cmd/sync: speed up sync for huge file (#4456)
- cmd/sync: use larger io-size when writing to backends (#4546)
- cmd/sync: skip listing the destination in force mode (#4602)
- cmd/sync: use HEAD instead of LIST for a single object (#4623)
- cmd/sync: limit the total size of each batch task from the manager (#4886)
- meta: add mount time in session info (#3987)
- meta: export meta backup metrics (#4480)
- meta: force compaction when there are too many slices in one chunk (#4573)
- meta: remove tail zero slice when compacting a chunk (#4789)
- meta/redis: reduce the memory usage when dumping large directories (#4481)
- meta/tkv & meta/sql: speedup the dump process using multiple threads (#4501)
- meta/badger: upgrade to v4 (#4564)
- chunk: prefetch the whole block only on successful partial read (#4658)
- chunk: delete leaked objects if it's already deleted by other goroutines (#4748)
- vfs: fuse ops metric supports showing method label (#4554, #4560)
- vfs: add errno label to fuse ops metric (#4656)
- object/tos: adjust min part size from 5 MiB to 4 MiB (#4688)
- object/ceph: allow delete when ceph cluster is full (#4131)
- object/ceph: speedup list for ceph (#4218)
- object/ceph: set default path for admin socket and log file (#4608)
- hadoop: support human friendly values for options (#4655, #4813)
- hadoop: support using
;
to separate values for juicefs.users and juicefs.group (#4724) - deps: upgrade golang.org/x/net to 0.23 (#4911)
Bugfix
- cmd/gateway: fix ListMultipartUploads sorting rule and support delimiter (#4297)
- cmd/gateway: fix HEAD directory returns 200 but expect 404 (#4486)
- cmd/gateway: fix the issue that memory is not released immediately after object downloaded (#4507)
- cmd/gateway: fix empty bucket display exception (#4875)
- cmd/gateway: fix the size of JfsObjects.CompleteMultipartUpload (#4893)
- cmd/gateway: fix listing objects with symbolic links (#4919)
- cmd/sync: fix the issue that empty files are always synchronized (#4223)
- cmd/sync: execute chmod after chown to avoid setuid/setgid failure (#4418)
- cmd/sync: fix the issue that setuid/setgid/sticky bits are not set when using
jfs
protocol (#4419) - cmd/sync: return the error of listing objects (#4879)
- cmd/sync: fix the issue that manager's IP address is never set in cluster mode (#4939)
- cmd/clone: set the attr should be done after checking permissions (#4745)
- cmd/clone: fix source mode check (#4856)
- cmd/debug: add a timeout when collecting the information (#4824)
- meta: fix the issue that
df
may see negative values (#4591) - meta: fix the error message in quota that always shows inode 0 (#4741)
- meta: fix the protection check for compaction (#4901)
- meta: fix the error code of baseMeta.Open (#4920)
- meta/redis: fix some transactions that are not wrapped by multi/exec (#4801)
- meta/sql: use upsert in Write() to avoid possible deadlock (#4529)
- meta/postgreSQL: fix the address parse (#4731)
- vfs: fix the issue that file entry may still be returned if it's deleted during readdir (#4453)
- vfs: fix the issue that file length may not be updated immediately after fallocate & copyfilerange (#4427)
- vfs: fix the race of getting file length (#4876)
- vfs: fix the issue that plock may be unlocked by mistake (#4910)
- hadoop: make libjfs singleton to avoid possible jnr weakhashmap infinity loop (#4407)
- hadoop: fix guid lookupGroup (#4532)
- hadoop: use auto-increment fs handle to avoid possible conflict (#4411)
- hadoop: fix the issue that
dfsio
does not work becausemapreduce.output.fileoutputformat.compress
is true (#4646)