apache/opendal v0.4.0

on GitHub

What's Changed

• refactor: Make read return BytesStream instead by @Xuanwo in #192
• feat(services/azblob): Implement list support by @D2Lark in #193
• refactor: Make write return BytesSink instead by @Xuanwo in #194
• feat: Implement io_util like into_sink and into_stream by @Xuanwo in #197
• feat(io_util): Implement observer for sink and stream by @Xuanwo in #198
• RFC-0191: Async Streaming IO by @Xuanwo in #191
• docs: Add docs for all newly added public functions by @Xuanwo in #199
• refactor: New public API design by @Xuanwo in #201
• deps: Bump to reqsign to fix s3 url encode issue by @Xuanwo in #202
• RFC-0203: Remove Credential by @Xuanwo in #203
• refactor: Adopt io::Result instead by @Xuanwo in #204
• refactor: Rollback changes around async streaming io by @Xuanwo in #205
• docs: Add docs for public types by @Xuanwo in #206
• refactor: Refactor behavior tests with macro_rules by @Xuanwo in #207
• Bump to version 0.4.0 by @Xuanwo in #208

Full Changelog: v0.3.0...v0.4.0

Refer to Upgrade From v0.3 to v0.4 section for more upgrade details.

From v0.3 to v0.4

OpenDAL introduces many breaking changes in v0.4.

Object::reader() is not AsyncSeek anymore

Since v0.4, Object::reader() will return impl BytesRead instead of Reader that implements AsyncRead and AsyncSeek. Users who want AsyncSeek please wrapped with opendal::io_util::seekable_read:

use opendal::io_util::seekable_read;

let o = op.object("test");
let mut r = seekable_read(&o, 10..);
r.seek(SeekFrom::Current(10)).await?;
let mut bs = vec![0;10];
r.read(&mut bs).await?;

Use RangeBounds instead

Since v0.4, the following APIs will be removed.

• Object::limited_reader(size: u64)
• Object::offset_reader(offset: u64)
• Object::range_reader(offset: u64, size: u64)

Instead, OpenDAL is providing a more general range_reader powered by RangeBounds:

pub async fn range_reader(&self, range: impl RangeBounds<u64>) -> Result<impl BytesRead>

Users can use their familiar rust range syntax:

let r = o.range_reader(1024..2048).await?;

Return io::Result instead

Since v0.4, all functions in OpenDAL will return std::io::Result instead.

Please check via std::io::ErrorKind directly:

use std::io::ErrorKind;

if let Err(e) = op.object("test_file").metadata().await {
    if e.kind() == ErrorKind::NotFound {
        println!("object not exist")
    }
}

Removing Credential

Since v0.4, Credential has been removed, please use the API provided by Builder directly.

builder.access_key_id("access_key_id");
builder.secret_access_key("secret_access_key");

Write returns BytesWriter instead

Since v0.4, Accessor::write will return a BytesWriter instead accepting a BoxedAsyncReader.

Along with this change, the old Writer has been replaced by a new set of write functions:

pub async fn write(&self, bs: impl AsRef<[u8]>) -> Result<()> {}
pub async fn writer(&self, size: u64) -> Result<impl BytesWrite> {}

Users can write into an object more easily:

let _ = op.object("path/to/file").write("Hello, World!").await?;

io_util replaces readers

Since v0.4, mod io_util will replace readers. In io_utils, OpenDAL provides helpful functions like:

• into_reader: Convert BytesStream into BytesRead
• into_sink: Convert BytesWrite into BytesSink
• into_stream: Convert BytesRead into BytesStream
• into_writer: Convert BytesSink into BytesWrite
• observe_read: Add callback for BytesReader
• observe_write: Add callback for BytesWrite

New type alias

For better naming, types that OpenDAL returns have been renamed:

• AsyncRead + Unpin + Send => BytesRead
• BoxedAsyncReader => BytesReader
• AsyncWrite + Unpin + Send => BytesWrite
• BoxedAsyncWriter => BytesWriter
• ObjectStream => ObjectStreamer