GoogleCloudPlatform/gcsfuse v2.0.0

Gcsfuse v2.0.0

on GitHub

File caching: A client-based local file cache that lets repeat file reads be served from a faster cache storage of your choice, such as a Local SSD, Persistent Disk, or in-memory /tmpfs. It significantly improves your read performance on subsequent reads/epochs.

• New config-file flags

      cache-dir: (default=””)
      file-cache:
          max-size-mb: (default=-1) 
          cache-file-for-range-read: (default=false)
  

• Behavior:
  • cache-dir: The file-cache feature is disabled by default. To enable it, pass a directory to cache-dir.
  • max-size-mb: This is used to limit the total capacity that the Cloud Storage FUSE cache can use within the specified cache directory. The eviction of cached data is based on a least recently used (LRU) algorithm. By default, a value of -1 signifies all available capacity in the specified directory.
  • cache-file-for-range-read: Enable file cache for Random & partial reads: Support for asynchronous cache ingestion of files on offset reads through cache-file-for-range-read.

Updates to metadata-cache (stat & type caches) feature

• Deprecated CLI flags
  • stat-cache-capacity is replaced with metadata-cache:stat-cache-max-size-mb.
  • stat-cache-ttl & type-cache-ttl is replaced with metadata-cache:ttl-secs.
• New config-file flags

     metadata-cache: 
           stat-cache-max-size-mb: (default=32)
           ttl-secs: (default=60)
           type-cache-max-size-mb: (default=4)
  

• Behavior changes
  • type-cache-max-size-mb: Type-cache size limit can now be configured using this configuration, at per-directory level.
  • stat-cache-max-size-mb: The stat-cache size limit is now configured as memory size (in MiB) using this configuration instead of capacity (count of entries). Also, stat-cache is now configured at mount-level as opposed to bucket-level.
  • ttl-secs: stat-cache and type-cache TTLs have been merged into a single-TTL value, configurable via this configuration. Also, this new TTL is set as an integer multiple of seconds, as opposed to fine-grained duration (such as 1s500ms with the old flags).
  • To get the best performance, stat-cache-max-size-mb, ttl-secs and type-cache-max-size-mb can be individually set to -1 for unlimited size/ttl limits. Setting any of them to 0 disables the corresponding cache.
  • Use of deprecated flags will give warning, but for backward compatibility, their values will still heuristically be converted to the new flags and will be used. If both new and old flags are set, old flags are ignored.

Managed folders compatibility

• GCSFuse respects the managed folder IAM permissions in the bucket mounted with GCSFuse.
• Pass --implicit-dir flag to view managed folders.

Memory Improvements

• The average memory usage of stat-cache is now reduced by up to 40%.

Bug fixes and improvements

• Fix concurrent list directory and file creation
  • #1726
  • #1603
• Improvements in logging
  • Added log for successful/failed mounting on stdout when log-file is set in non-foreground mode.

Removed --experimental-local-file-cache flag which used for an old experimental local file cache feature.

Upgraded Dependencies for better stability and reliability. (#1744, #1746)