github apache/opendal v0.54.0
on GitHub

latest releases: bindings/go/v0.1.14-rc.1, v0.54.1-rc.1
one month ago

Upgrade Notes

Rust Core Public API

RFC-6189: Remove Native Blocking Support

OpenDAL v0.54 implements RFC-6189, which removes all native blocking support in favor of using block_on from async runtimes.

The following breaking changes have been made:

  • blocking::Operator can no longer be used within async contexts
  • Using blocking APIs now requires an async runtime
  • All Blocking* types have been moved to the opendal::blocking module

To migrate: 

- use opendal::BlockingOperator;
+ use opendal::blocking::Operator;

RFC-6213: Options Based API

OpenDAL v0.54 implements RFC-6213, which introduces options-based APIs for more structured and extensible operation configuration.

New APIs added:

  • read_options(path, ReadOptions)
  • write_options(path, data, WriteOptions)
  • list_options(path, ListOptions)
  • stat_options(path, StatOptions)
  • delete_options(path, DeleteOptions)

Example usage: 

// Read with options
let options = ReadOptions::new()
    .range(0..1024)
    .if_match("etag");
let data = op.read_options("path/to/file", options).await?;

// Write with options  
let options = WriteOptions::new()
    .content_type("text/plain")
    .cache_control("max-age=3600");
op.write_options("path/to/file", data, options).await?;

Remove stat_has_xxx and list_has_xxx APIs

All stat_has_* and list_has_* capability check APIs have been removed. Instead, check capabilities directly on the Capability struct: 

- if op.info().full_capability().stat_has_content_length() {
+ if op.info().full_capability().stat.content_length {
    // ...
}

Fix with_user_metadata signature

The signature of with_user_metadata has been changed. Please update your code accordingly if you use this method.

Services removed due to lack of maintainer

The following services have been removed due to lack of maintainers:

  • atomicserver
  • icloud
  • nebula_graph

If you need these services, please consider maintaining them or use alternative services.

HttpClientLayer replaces update_http_client

The Operator::update_http_client() method has been replaced by HttpClientLayer: 

- op.update_http_client(client);
+ op = op.layer(HttpClientLayer::new(client));

Expose presign_xxx_options API

New options-based presign APIs have been exposed: 

let options = PresignOptions::new()
    .expire(Duration::from_secs(3600));
    
let url = op.presign_read_options("path/to/file", options).await?;

Rust Core Raw API

Remove native blocking support

All native blocking implementations have been removed from the raw API. Services and layers no longer need to implement blocking-specific methods.

Bindings Java Breaking changes

Removed services

The following services have been removed:

  • Chainsafe service has been removed (PR-5744) - The service has been sunset.
  • libsql service has been removed (PR-5616) - Dead service removal.

Batch operations removed

PR-5393 removes the batch concept from OpenDAL. All batch-related operations and capabilities have been removed.

Capability changes

  • write_multi_align_size capability has been removed (PR-5322)
  • Added new shared capability (PR-5328)

Options-based API

New options classes have been introduced for structured operation configuration:

  • ReadOptions - for read operations
  • WriteOptions - for write operations
  • ListOptions - for list operations
  • StatOptions - for stat operations

Example usage: 

// Read with options
ReadOptions options = ReadOptions.builder()
    .range(0, 1024)
    .ifMatch("etag")
    .build();
byte[] data = operator.read("path/to/file", options);

// Write with options
WriteOptions options = WriteOptions.builder()
    .contentType("text/plain")
    .cacheControl("max-age=3600")
    .build();
operator.write("path/to/file", data, options);

Bindings Python

Breaking change: Native blocking API removed

OpenDAL has removed the native blocking API in the core. The Python binding's blocking API now uses an async runtime internally. This is a transparent change and should not affect most users, but:

  • The blocking API now requires an async runtime to be available
  • Performance characteristics may be slightly different
  • All blocking operations are now implemented as async operations running in a tokio runtime

Breaking change: Removed services

The following services have been removed due to lack of maintainers and users:

  • atomicserver - This service is no longer supported
  • icloud - This service is no longer supported
  • nebula_graph - This service is no longer supported

If you were using any of these services, you'll need to migrate to an alternative storage backend.

Breaking change: Chainsafe service removed

The Chainsafe service has been sunset and is no longer available.

What's Changed

Added

Changed

  • refactor(core/types)!: fix with_user_metadata signature by @meteorgan in #5960
  • refactor(!): Remove services lack of maintainers and users by @Xuanwo in #6263
  • refactor(services/moka)!: replace sync::Cache with future::Cache by @koushiro in #6270
  • refactor(bindings/go): Restructure FFI system with type-safe wrapper by @yuchanns in #6268
  • refactor: Migrate redis from adapter::kv to Access instead by @Xuanwo in #6291
  • refactor: Migrate moka from adapter::typed_kv to Access instead by @Xuanwo in #6300
  • refactor: Migrate memory service to implment Access directly by @Xuanwo in #6301
  • refactor: Migrate services cacache to implement Access by @Xuanwo in #6303
  • refactor!: Remove stat_has_xxx and list_has_xxx by @Xuanwo in #6313
  • refactor(services/fs): extract implementation to core by @erickguan in #6317
  • refactor: Migrate mini_moka service to implement Access directly by @meteorgan in #6316
  • refactor: remove uuid dependency when creating a temp path by @erickguan in #6324
  • refactor(layers/logging): Don't trigger logigng in heavy IO path by @Xuanwo in #6343
  • refactor: Migrate dashmap service to implement Access directly by @meteorgan in #6344

Fixed

  • fix: java bug in list with delete option test by @geruh in #6257
  • fix(nodejs): esmodule and commonjs support by @kingsword09 in #6266
  • fix(gcs): headers missing in XML multipart API and incorrect x-goog-acl header values in XML API by @wlinna in #6275
  • fix(bindings/nodejs): update nodejs and deno bench by @kingsword09 in #6286
  • fix(hdfs): fix infinite loop in write for HDFS failure by @oven-yang in #6295
  • fix(nodejs): test stat with version by @kingsword09 in #6307
  • fix(bindings/python): Fix Writer doesn't throw correct error code by @Xuanwo in #6315
  • fix(python): correctly calculate end bound using offset + size instead of size directly by @kingsword09 in #6314
  • fix(fs/ftp/hdfs): correct tmp_path generation for append operations by @kingsword09 in #6327
  • fix(dav-server): Fix create_dir to create nested directories by @sqlpxc in #6321
  • fix(service/fs): handle if_not_exists flag to raise ConditionNotMatch error by @kingsword09 in #6326
  • fix(services/fs): Avoid creating partial files by @Xuanwo in #6336
  • fix(bindings/nodejs): ListOptions test list with deleted by @kingsword09 in #6335
  • fix(bindings/go): ffi calls use after free by @yuchanns in #6380
  • fix(services/azdls): Fix append not handled correctly while offset==0 by @Xuanwo in #6393

Docs

  • docs: Remove deprecated APIs and polish docs for public APIs by @Xuanwo in #6220
  • docs(services/hdfs_native): fix outdated capabilities and config option name by @kezhuw in #6224
  • docs: Add CLAUDE docs to make AI Agents happy by @Xuanwo in #6299
  • docs: Polish claude file after some experiments by @Xuanwo in #6302
  • docs: Add upgrade guide for opendal's 0.54 release by @Xuanwo in #6382

CI

  • ci: Use expression syntax to avoid VS Code warnings. by @kingsword09 in #6284
  • ci: Disable failed CI until #6305 been fixed by @Xuanwo in #6306
  • chore(ci): use mlugg/setup-zig instead of archvied action by @assignUser in #6310
  • ci(bindings/go): simplify and improve Go bindings test infrastructure by @yuchanns in #6293
  • ci(services/compfs): add integrtation tests for compfs service by @meteorgan in #6319
  • chore(dependabot): update CI for dependabot PRs by @erickguan in #6411

Chore

  • chore: add deepwiki badge into readme to enable auto-refresh by @koushiro in #6200
  • chore: upgrade opentelemetry to 0.30.0 by @tisonkun in #6259
  • chore: Update bb8 to version 0.9.0 by @cryptomilk in #6127
  • chore(deps): bump uuid from 1.16.0 to 1.17.0 in /bin/oli by @dependabot[bot] in #6245
  • chore(bindings/go): update Go dependencies by @yuchanns in #6280
  • chore(deps): bump tokio from 1.45.0 to 1.45.1 in /bin/ofs by @dependabot[bot] in #6240
  • chore(metrics): add more docs about global instance of PrometheusLayer and FastmetricsLayer by @koushiro in #6308
  • chore: Add npm to dependabot by @shaonianche in #6318
  • chore(deps): bump astral-sh/setup-uv from 5 to 6 by @dependabot[bot] in #6241
  • chore(deps): update datafusion requirement from 47.0.0 to 48.0.0 in /integrations/object_store by @dependabot[bot] in #6332
  • chore(deps): bump @docusaurus/core from 3.6.1 to 3.8.1 in /website by @dependabot[bot] in #6334
  • chore(deps): bump compio from 0.14.0 to 0.15.0 in /core by @dependabot[bot] in #6331
  • chore(deps): bump clap from 4.5.38 to 4.5.40 in /bin/ofs by @dependabot[bot] in #6330
  • chore(deps): bump dirs from 5.0.1 to 6.0.0 in /bin/oli by @dependabot[bot] in #6329
  • chore: fix clippy warnings when using rust 1.88 by @koushiro in #6339
  • chore(dav-server): Add a test for creating nested directories by @sqlpxc in #6338
  • chore: bump msrv to v1.82.0 by @MrCroxx in #6348
  • chore(deps): bump crate-ci/typos from 1.31.1 to 1.34.0 by @dependabot[bot] in #6351
  • chore(deps): bump tokio from 1.45.0 to 1.45.1 in /bin/oli by @dependabot[bot] in #6353
  • chore(deps): bump quick-xml from 0.36.2 to 0.37.5 in /bin/oay by @dependabot[bot] in #6350
  • chore(deps): bump actions/checkout from 3 to 4 by @dependabot[bot] in #6356
  • chore(deps): bump semver from 7.6.3 to 7.7.2 in /website by @dependabot[bot] in #6352
  • chore(deps): bump toml from 0.8.22 to 0.8.23 in /bin/oli by @dependabot[bot] in #6361
  • chore(deps): bump mlugg/setup-zig from 2.0.1 to 2.0.3 by @dependabot[bot] in #6359
  • chore(deps): bump axios from 1.8.2 to 1.10.0 in /website by @dependabot[bot] in #6363
  • chore(deps): bump tokio from 1.45.0 to 1.45.1 in /bin/oay by @dependabot[bot] in #6358
  • chore(deps): bump logforth from 0.24.0 to 0.26.1 in /bin/ofs by @dependabot[bot] in #6355
  • chore(deps): bump the pyo3-dependencies group in /bindings/python with 2 updates by @dependabot[bot] in #6360
  • chore(deps): bump hdfs-native from 0.10.4 to 0.11.2 in /core by @dependabot[bot] in #6362
  • chore: update DEPENDENCIES and fater dependencies.py generate by @yihong0618 in #6374
  • chore(dependabot): update OpenDAL dependencies less frequently by @erickguan in #6384
  • chore(deps): bump tokio from 1.45.1 to 1.46.1 in /bin/oli by @dependabot[bot] in #6387
  • chore(deps): bump tokio from 1.45.1 to 1.46.1 in /bin/ofs by @dependabot[bot] in #6386
  • chore(deps): bump logforth from 0.24.0 to 0.26.1 in /bin/oay by @dependabot[bot] in #6385
  • chore(deps): bump clsx from 1.2.1 to 2.1.1 in /website by @dependabot[bot] in #6388
  • chore(deps): bump tokio from 1.45.0 to 1.46.1 in /core in the async-runtime group by @dependabot[bot] in #6390
  • chore(dependabot): exclude rand and getrandom crates by @erickguan in #6397
  • chore(github-actions): update 1password by @erickguan in #6405
  • chore(github-actions): revert 1password action update by @erickguan in #6406
  • chore(deps): bump nix from 0.29.0 to 0.30.1 in /bin/ofs by @dependabot[bot] in #6399
  • chore(deps): bump toml from 0.8.22 to 0.9.2 in /bin/oay by @dependabot[bot] in #6398
  • chore(deps): bump toml from 0.8.23 to 0.9.2 in /bin/oli by @dependabot[bot] in #6400
  • chore(deps): bump react and react-dom in /website by @dependabot[bot] in #6401
  • chore(deps): bump tower from 0.4.13 to 0.5.2 in /bin/oay by @dependabot[bot] in #6407
  • chore: Disable openssh build to allow python release by @Xuanwo in #6412

New Contributors

Full Changelog: v0.53.3...v0.54.0

Check out latest releases or
releases around apache/opendal v0.54.0

Don't miss a new opendal release

NewReleases is sending notifications on new releases.

Get notifications