github spf13/afero v1.7.0
v1.7.0: Add experimental GCS support

latest releases: v1.11.0, v1.10.0, v1.9.5...
2 years ago

Add experimental GCS support in Afero. Experimental because the CI infra of afero does not test with real GCS buckets.

Limitations:

  • No Chmod support - The GCS ACL could probably be mapped to *nix style permissions but that would add another level of complexity and is ignored in this version.
  • No Chtimes support - Could be simulated with attributes (gcs a/m-times are set implicitly) but that's is left for another version.
  • NOTE: Not thread safe - Also assumes all file operations are done through the same instance of the GcsFs. File operations between different GcsFs instances are not guaranteed to be consistent.

Performance implications

  • Sequential reads are performant
  • Sequential writes are performant.
  • Seek + Read or ReadAt is performant after the initial seek. (produces a warning)
  • Alternating reads/writes to the same file handler are highly inefficient. To get consistent FS behavior using an API that separates readers and writers we close any open readers before an write as well close open writers before a read (ensure the data is committed).
  • Seek + Write such as WriteAt, Truncate, Seek+Write will work as expected but with significant overhead. Doing a seek + write will in effect download the old file/object, overlay it with the new writes and save it back. This is done in a streaming fashion so large files will not clog the memory but will trigger a full download and upload of the file/object.

Don't miss a new afero release

NewReleases is sending notifications on new releases.