New Features:
- Kernel-list-cache:
- List responses, that happen as a part of a readdir
operation, are cached in the kernel page cache. This can significantly speed
up AI/ML training runs, which do full directory listing first, by serving
repeat ListObjects calls locally from the kernel page cache. - Cache invalidation is controlled via the --
kernel-list-cache-ttl-secs
cli
flag orfile-system:kernel-list-cache-ttl-secs
config flag , where a value
of:- 0 means disabled. This is the default value.
- valid positive - represents the ttl (in seconds) to keep the directory
list response in the kernel page-cache. - -1 to bypass a TTL expiration and serve the list response from the cache
whenever it's available.
- List responses, that happen as a part of a readdir
Enhancements:
- Allow parallel lookups of files:
- Allows parallel lookup/access of files under the same directory.
- Before this release, if an application accessed two files
/gcsfuse/mount/a.txt
&/gcsfuse/mount/b.txt
in parallel, then access was
serialized (both at Kernel's FUSE driver layer and GCSFuse). - With this release, access is parallelized improving read performance up to
18x when reading 100K files using 50 threads.
Dependency Upgrades / CVE fixes:
- No dependency upgrades or CVE fixes.
What's Changed
- Moving auth and config packages to stretchr/testify by @sethiay in #1918
- BucketType method to locally store bucketType in variable by @Tulsishah in #1896
- [PR1] allow parallel lookups by @sethiay in #1866
- Include code-coverage badge in README by @kislaykishore in #1933
- Update .codecov.yml to use the PRs base as reference by @kislaykishore in #1934
- [PR2] Composite tests for Parallel dirops test. by @sethiay in #1906
- Move to stretchr testify in ratelimit & util packages by @sethiay in #1922
- Add a Cobra root command. by @kislaykishore in #1908
- [PR3] E2E tests for parallel dirops by @sethiay in #1907
- Add timeout to the coverage workflow by @kislaykishore in #1940
- Metadata-prefetch (aka Recursive listing or ls -R during mount) by @gargnitingoogle in #1930
- [PR1] Support to keep ReadDir response in kernel cache by @raj-prince in #1897
- Implement method to return bucket type from server by @Tulsishah in #1898
- Disable the "changes" feature of codecov by @kislaykishore in #1952
- Remove some flagset to decrease timeout for e2e tests by @Tulsishah in #1951
- Mark "metadata-prefetch-on-mount" as experimental by @gargnitingoogle in #1957
Full Changelog: v2.1.0...v2.2.0