github apache/opendal v0.47.0
on GitHub

latest releases: v0.47.1-rc.2, v0.47.1-rc.1
7 days ago

Core - Upgrade to v0.47

Public API

Reader into_xxx APIs

Since v0.47, Reader's into_xxx APIs requires async and returns Result instead. 

- let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048);
+ let r = op.reader("test.txt").await?.into_futures_async_read(1024..2048).await?;

Affected API includes:

  • Reader::into_futures_async_read
  • Reader::into_bytes_stream
  • BlockingReader::into_std_read
  • BlockingReader::into_bytes_iterator

Raw API

Bring Streaming Read Back

As explained in core: Bring Streaming Read Back, we do need read streaming back for better performance and low memory usage.

So our oio::Read changed back to streaming read instead: 

trait Read {
-  async fn read(&self, offset: u64, size: usize) -> Result<Buffer>;
+  async fn read(&mut self) -> Result<Buffer>;
}

All services and layers should be updated to meet this change.

Java Binding - Upgrade to v0.46

Breaking change

PR-4641 renames async Operator to AsyncOperator and BlockingOperator to Operator.

New features

PR-4626 implements OperatorInputStream and OperatorOutputStream which implements Java's core IO abstractions InputStream and OutputStream. Users can now read/write bytes streamlined without loading/preparing the bytes fully in memory.

What's Changed

Added

  • feat(core/types): change oio::BlockingReader to Arc by @hoslo in #4577
  • fix: format_object_meta should not require metakeys that don't exist by @rebasedming in #4582
  • feat: add checksums to MultiPartComplete by @JWackerbauer in #4580
  • feat(doc): update object_store_opendal README by @hanxuanliang in #4606
  • feat(services/aliyun-drive): support AliyunDrive by @yuchanns in #4585
  • feat(bindings/python): Update type annotations by @3ok in #4630
  • feat: implement OperatorInputStream and OperatorOutputStream by @tisonkun in #4626
  • feat(bench): add buffer benchmark by @zjregee in #4603
  • feat: Add Executor struct and Execute trait by @Xuanwo in #4648
  • feat: Add executor in OpXxx and Operator by @Xuanwo in #4649
  • feat: Implement and refactor concurrent tasks for multipart write by @Xuanwo in #4653
  • feat(core/types): blocking remove_all for object storage based services by @TennyZhuang in #4665
  • feat(core): Streaming reading while chunk is not set by @Xuanwo in #4658
  • feat(core): Add more context in error context by @Xuanwo in #4673
  • feat: init ovfs by @zjregee in #4652
  • feat: Implement retry for streaming based read by @Xuanwo in #4683
  • feat(core): Implement TimeoutLayer for concurrent tasks by @Xuanwo in #4688
  • feat(core): Add reader size check in complete reader by @Xuanwo in #4690
  • feat(core): Azblob supports azure workload identity by @Xuanwo in #4705

Changed

  • refactor(core): Align naming for AccessorDyn by @morristai in #4574
  • refactor(core): core doesn't expose invalid input error anymore by @Xuanwo in #4632
  • refactor(core): Return unexpected error while content incomplete happen by @Xuanwo in #4633
  • refactor(core): Change Read's behavior to ensure it reads the exact size of data by @Xuanwo in #4634
  • refactor(bin/ofs): Fuse API by @ho-229 in #4637
  • refactor(binding/java)!: rename blocking and async operator by @tisonkun in #4641
  • refactor(core): Use concurrent tasks to refactor block write by @Xuanwo in #4692
  • refactor(core): Migrate RangeWrite to ConcurrentTasks by @Xuanwo in #4696

Fixed

  • fix(devcontainer/post_create.sh): change pnpm@stable to pnpm@latest by @GG2002 in #4584
  • fix(bin/ofs): privileged mount crashes when external umount by @ho-229 in #4586
  • fix(bin/ofs): ofs read only mount by @ho-229 in #4602
  • fix(raw): Allow retrying request while decoding response failed by @Xuanwo in #4612
  • fix(core): return None if metadata unavailable by @NKID00 in #4613
  • fix(bindings/python): Use abi3 and increase MSPV to 3.11 by @Xuanwo in #4623
  • fix: Fetch the content length while end_bound is unknown by @Xuanwo in #4631
  • fix: ofs write behavior by @ho-229 in #4617
  • fix(core/types): remove_all not work under object-store backend by @TennyZhuang in #4659
  • fix(ofs): Close file during flush by @Xuanwo in #4680
  • fix(core): RetryLayer could panic when other threads raises panic by @Xuanwo in #4685
  • fix(core/prometheus): Fix metrics from prometheus not correct for reader by @Xuanwo in #4691
  • fix(core/oio): Make ConcurrentTasks cancel safe by only pop after ready by @Xuanwo in #4707

Docs

CI

  • ci: upgrade typos to 1.21.0 and ignore changelog by @hezhizhen in #4601
  • ci: Disable jfrog webdav tests for it's keeping failed by @Xuanwo in #4607
  • ci: use official typos github action by @shoothzj in #4635
  • build(deps): upgrade crc32c to 0.6.6 for nightly toolchain by @tisonkun in #4650

Chore

New Contributors

Full Changelog: v0.46.0...v0.47.0

Check out latest releases or
releases around apache/opendal v0.47.0

Don't miss a new opendal release

NewReleases is sending notifications on new releases.

Get notifications