github juicedata/juicefs v1.2.0

latest releases: v1.2.1, v1.1.4, v0.2.1...
5 months ago

This is the third major release of JuiceFS, and is an LTS version as well that will be maintained for 24 months.
Starting from v1.1 it has 464 commits from 44 contributors, thanks to @zhijian-pro @davies @zhoucheng361 @SandyXSD @jiefenghuang @polyrabbit @CaitinChen @yuhr123 @Hexilee @tangyoupeng @YunhuiChen @eryugey @timfeirg @Suave @solracsf @kyungwan-nam @xiaogaozi @1ambd4 @stefanb @XDTD @occupyhabit @menwenjun @li1213987842 @testwill @fengleng @deferdeter @camcui @uvletter @T-TRz879 @yuchen-sun @TechQuery @reAsOn2010 @mtnk2005 @kanemathers @JoJossd @sjp00556 @CodingPoeta @abserari @AChangFeng @feeyman @l0wl3vel @chnliyong @JoyLiuc @prOOrc!

Highlights

New

  • Added a new command compact to trigger compaction on a specified path (#4337)
  • Supports using NFS as object storage (#3823, #4101, #4105, #4111, #4147, #4163, #4718)
  • Supports using Dragonfly as object storage (#4057)
  • Supports using Bunny as object storage (#4719)
  • cmd: add a new option --log-id to append user-specified ID in log (#4040)
  • cmd/mount: supports restarting mount process gracefully (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555, #4792, #4823, etc.)
  • cmd/format: add a new option --enable-acl to enable POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
  • cmd/mount: add a new option --skip-dir-mtime to eliminate burst mtime updates of a directory (#4189)
  • cmd/mount: add a new option --upload-hours to limit the period for background uploads (#4250)
  • cmd/mount: add a new option --cache-expire to set expired time for cached blocks (#4255, #4365, #4434)
  • cmd/mount: add a new option --custom-labels to set labels for metrics (#4312, #4612)
  • cmd/mount: add a new environment variable JFS_MOUNT_TIMEOUT to adjust timeout for checking mount point ready (#4219)
  • cmd/mount: add a new option max-stage-write to enable concurrent write to both stage files and object storage in writeback mode (#4743)
  • cmd/gateway: supports IAM and events notify (#4030, #4593, #4622, #4845, #4891)
  • cmd/gateway&webdav: add a new option --background to support daemon running (#3993, #4631, #4708, #4915)
  • cmd/gateway: add a new option --domain to support virtual-host-style requests (#4462)
  • cmd/warmup: add a new option --evict to manually evict cached blocks (#4370)
  • cmd/warmup: and a new option --check to check the ratio of already warmed blocks (#4370)
  • cmd/dump: add a new option --fast=false to reduce memory usage with a slower dump process (#4253)
  • cmd/dump: add a new option --skip-trash to skip the trash directory when backing up metadata (#4479, #4563)
  • cmd/dump: add a new option --threads to adjust speed for dump (#4581)
  • cmd/sync: add a new option --inpace to put file in-place, eliminating the rename operation (#4104)
  • cmd/sync: supports two stars ** and three stars *** to match all files recursively (#4422, #4448, #4460, #4468, #4474)
  • cmd/sync: add a new option --match-full-path to match the filter against full path (#4492)
  • cmd/sync: add new options max-size, min-size, max-age and min-age to filter files based on size and age (#4912, 7491e33, #4913)
  • cmd/sync: add p8s metrics to better monitor the sync progress (#4119)
  • meta: supports Open File Description (OFD) lock (#4083)
  • chunk: disk cache switch to consistent hash and add bad disk detection (#4543, #4749, #4780, #4784, #4871, #4848)
  • object: add query string to disable md5 and crc verification when s3 compatible storage does not support them (#4606)
  • object: add a new environment variable JFS_S3_VHOST_STYLE=1 to enable vhost style (#4697)
  • hadoop: add push-labels option to set custom labels for metrics (#4312)

Changed

  • cmd: support human friendly values for options (#4246, #4558, #4567, #4592, #4641, #4671)
  • cmd/mount: only use writeback_cache to enable fuse writeback (#4632)
  • cmd/gateway: judge loop symbolic links rather than hard-code deep limit (#4045)
  • cmd/gateway: create MinioMetaBucket when it doesn't exist (#4383)
  • cmd/gateway: create .minio.sys directory even if multi-buckets are not enabled (#4367)
  • cmd/dump: use a temporary file during dumping (#4767)
  • cmd/sync: limit the maximum memory size for multiple uploads (#4135)
  • cmd/sync: speed up sync for huge file (#4456)
  • cmd/sync: use larger io-size when writing to backends (#4546)
  • cmd/sync: skip listing the destination in force mode (#4602)
  • cmd/sync: use HEAD instead of LIST for a single object (#4623)
  • cmd/sync: limit the total size of each batch task from the manager (#4886)
  • meta: add mount time in session info (#3987)
  • meta: export meta backup metrics (#4480)
  • meta: force compaction when there are too many slices in one chunk (#4573)
  • meta: remove tail zero slice when compacting a chunk (#4789)
  • meta/redis: reduce the memory usage when dumping large directories (#4481)
  • meta/tkv & meta/sql: speedup the dump process using multiple threads (#4501)
  • meta/badger: upgrade to v4 (#4564)
  • chunk: prefetch the whole block only on successful partial read (#4658)
  • chunk: delete leaked objects if it's already deleted by other goroutines (#4748)
  • vfs: fuse ops metric supports showing method label (#4554, #4560)
  • vfs: add errno label to fuse ops metric (#4656)
  • object/tos: adjust min part size from 5 MiB to 4 MiB (#4688)
  • object/ceph: allow delete when ceph cluster is full (#4131)
  • object/ceph: speedup list for ceph (#4218)
  • object/ceph: set default path for admin socket and log file (#4608)
  • hadoop: support human friendly values for options (#4655, #4813)
  • hadoop: support using ; to separate values for juicefs.users and juicefs.group (#4724)
  • deps: upgrade golang.org/x/net to 0.23 (#4911)

Bugfix

  • cmd/gateway: fix ListMultipartUploads sorting rule and support delimiter (#4297)
  • cmd/gateway: fix HEAD directory returns 200 but expect 404 (#4486)
  • cmd/gateway: fix the issue that memory is not released immediately after object downloaded (#4507)
  • cmd/gateway: fix empty bucket display exception (#4875)
  • cmd/gateway: fix the size of JfsObjects.CompleteMultipartUpload (#4893)
  • cmd/gateway: fix listing objects with symbolic links (#4919)
  • cmd/sync: fix the issue that empty files are always synchronized (#4223)
  • cmd/sync: execute chmod after chown to avoid setuid/setgid failure (#4418)
  • cmd/sync: fix the issue that setuid/setgid/sticky bits are not set when using jfs protocol (#4419)
  • cmd/sync: return the error of listing objects (#4879)
  • cmd/sync: fix the issue that manager's IP address is never set in cluster mode (#4939)
  • cmd/clone: set the attr should be done after checking permissions (#4745)
  • cmd/clone: fix source mode check (#4856)
  • cmd/debug: add a timeout when collecting the information (#4824)
  • meta: fix the issue that df may see negative values (#4591)
  • meta: fix the error message in quota that always shows inode 0 (#4741)
  • meta: fix the protection check for compaction (#4901)
  • meta: fix the error code of baseMeta.Open (#4920)
  • meta/redis: fix some transactions that are not wrapped by multi/exec (#4801)
  • meta/sql: use upsert in Write() to avoid possible deadlock (#4529)
  • meta/postgreSQL: fix the address parse (#4731)
  • vfs: fix the issue that file entry may still be returned if it's deleted during readdir (#4453)
  • vfs: fix the issue that file length may not be updated immediately after fallocate & copyfilerange (#4427)
  • vfs: fix the race of getting file length (#4876)
  • vfs: fix the issue that plock may be unlocked by mistake (#4910)
  • hadoop: make libjfs singleton to avoid possible jnr weakhashmap infinity loop (#4407)
  • hadoop: fix guid lookupGroup (#4532)
  • hadoop: use auto-increment fs handle to avoid possible conflict (#4411)
  • hadoop: fix the issue that dfsio does not work because mapreduce.output.fileoutputformat.compress is true (#4646)

Don't miss a new juicefs release

NewReleases is sending notifications on new releases.