apache/opendal v0.43.0 on GitHub

Upgrade Note

Rust Core

Public API

List Recursive

After RFC-3526: List Recursive landed, we have changed the list API to accept recursive instead of delimiter:

Users will need to change the following usage:

op.list_with(path).delimiter("") -> op.list_with(path).recursive(true)
op.list_with(path).delimiter("/") -> op.list_with(path).recursive(false)

delimiter other than "" and "/" is not supported anymore.

Stat a dir path

After RFC: List Prefix landed, we have changed the behavior of stat a dir path:

Here are the behavior list:

Case	Path	Result
stat existing dir	`abc/`	Metadata with dir mode
stat existing file	`abc/def_file`	Metadata with file mode
stat dir without `/`	`abc/def_dir`	Error `NotFound` or metadata with dir mode
stat file with `/`	`abc/def_file/`	Error `NotFound`
stat not existing path	`xyz`	Error `NotFound`

Services like s3, azblob can handle stat("abc/") correctly by check if there are objects with prefix abc/.

Raw API

Lister Align

We changed our internal lister implementation to align with the list public API for better performance and readability.

trait Page => List
struct Pager => Lister
trait BlockingPage => BlockingList
struct BlockingPager => BlockingLister

Every call to next will return an entry instead a page of entries. Also, we changed our async list api into poll based instead of async_trait.

Java binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Node.js binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Python binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

C binding

There are no API changes.

What's Changed

Added

feat(bindings/C): Add opendal_operator_rename and opendal_operator_copy by @jiaoew1991 in #3517
feat(binding/python): Add new API to convert between AsyncOperator and Operator by @Zheaoli in #3514
feat: Implement RFC-3526: List Recursive by @Xuanwo in #3556
feat(service): add alluxio rest api support by @hoslo in #3564
feat(bindings/python): add OPENDAL_DISABLE_RANDOM_ROOT support by @Justin-Xiang in #3550
feat(core): add Alluxio e2e test by @hoslo in #3573
feat(service): alluxio support write by @hoslo in #3566
feat(bindings/nodejs): add retry layer by @suyanhanx in #3484
RFC: Concurrent Stat in List by @morristai in #3574
feat(service/hdfs): enable rename in hdfs service by @qingwen220 in #3592
feat: Improve the read_to_end perf and add benchmark vs_fs by @Xuanwo in #3617
feat: Add benchmark vs aws sdk s3 by @Xuanwo in #3620
feat: Improve the performance of s3 services by @Xuanwo in #3622
feat(service): support b2 by @hoslo in #3604
feat(core): Implement RFC-3574 Concurrent Stat In List by @morristai in #3599
feat: Implement stat dir correctly based on RFC-3243 List Prefix by @Xuanwo in #3651
feat(bindings/nodejs): Add capability support by @suyanhanx in #3654
feat: disable ftp for python and java binding by @ZutJoe in #3659
feat(bindings/nodejs): read/write stream by @suyanhanx in #3619

Changed

refactor(services/persy): migrate tot test planner by @G-XD in #3476
refactor(service/etcd): Add EtcdConfig to implement ConfigDeserializer by @Xuxiaotuan in #3543
chore(service/tikv): rename Backend to TikvBackend by @caicancai in #3545
refactor(services/azblob): add AzblobConfig by @acehinnnqru in #3553
refactor(services/cacache): migrate to test planner by @G-XD in #3568
refactor(services/sled): migrate to test planner by @G-XD in #3569
refactor(services/webhdfs): migrate to test planner by @G-XD in #3578
refactor(core): Rename all Page to List by @Xuanwo in #3589
refactor: Change List API into poll based and return one entry instead by @Xuanwo in #3593
refactor(services/tikv): migrate to test planner by @G-XD in #3587
refactor(service/redis): Migrate task to new task planner by @sunheyi6 in #3374
refactor(oio): Polish IncomingAsyncBody::bytes by @Xuanwo in #3621
refactor(services/rocksdb): migrate to test planner by @G-XD in #3636
refactor(services/azfile): Check if dir exists before create by @ZutJoe in #3652
refactor: Polish concurrent list by @Xuanwo in #3658

Fixed

fix(bindings/python): Fix the test command in doc by @Justin-Xiang in #3541
docs(bindings/python): Fix the test command in doc by @Justin-Xiang in #3561
fix(ci): try enable corepack before setup-node action by @suyanhanx in #3609
fix(service/hdfs): enable hdfs append support by @qingwen220 in #3600
fix(ci): fix setup node by @suyanhanx in #3611
fix(core): Path in remove not normalized by @Xuanwo in #3671
fix(core): Build with redis features and Rust < 1.72 by @vincentdephily in #3683

Docs

docs: Add questdb in users list by @caicancai in #3532
docs: Add macos specific doc updates for hdfs by @shbhmrzd in #3559
docs(bindings/java): add basic usage in README by @caicancai in #3534
doc: add 0.42.0 release link to download.md by @silver-ymz in #3598

CI

ci(services/libsql): add rust test threads limit by @G-XD in #3540
ci(services/redb): migrate to test planner by @suyanhanx in #3518
ci: Disable libsql behavior test until we or upstream address them by @Xuanwo in #3552
ci: Add new Python binding reviewer by @Zheaoli in #3560
ci(bindings/nodejs): add aarch64 build support by @suyanhanx in #3567
ci(planner): Polish the workflow planner code by @Zheaoli in #3570
ci(core): Add dry run for rc tags by @Xuanwo in #3624
ci: Disable persy until it has been fixed by @Xuanwo in #3631
ci: Calling cargo to make sure rust has been setup by @Xuanwo in #3633
ci: Fix etcd with tls and auth failed to start by @Xuanwo in #3637
ci(services/etcd): Use ALLOW_NONE_AUTHENTICATION as workaround by @Xuanwo in #3638
ci: dry run publish on rc tags for python binding by @everpcpc in #3645
ci: Add java linux arm64 build by @Xuanwo in #3660
ci(java/binding): Use zigbuild for glibc 2.17 support by @Xuanwo in #3664
ci(bindings/python): remove aarch support by @G-XD in #3674

Chore

chore(servies/sftp): Upgrade openssh-sftp-client to 0.14 by @sd44 in #3538
chore(docs): add cpp binding in README by @cjj2010 in #3546
chore(service/sqlite): fix typo on sqlite by @caicancai in #3549
chore(bindings/C): resolve doxygen warnings by @sd44 in #3572
chore: removed dotenv in bindings/nodejs/index.js by @AlexVCS in #3579
chore: update opentelemetry to v0.21.x by @jtescher in #3580
chore: Add cpp binding Google style clang-format && format the code by @JackDrogon in #3581
chore: bump suppaftp version to 5.2 by @oowl in #3590
chore(ci): fix artifacts path for publish pypi by @everpcpc in #3597
chore: Code cleanup to make rust 1.74 happy by @Xuanwo in #3602
chore: Fix raw::tests been excluded unexpectedly by @Xuanwo in #3623
chore: Bump dpes and remove native-tls in mysql-async by @Xuanwo in #3627
chore(core): Have mysql_async use rustls instead of native-tls by @amunra in #3634
chore: Polish docs for Capability by @Xuanwo in #3635
chore: Bump reqsign to 0.14.4 for jsonwebtoken by @Xuanwo in #3644
chore(ci): nodejs binding publish dry run by @suyanhanx in #3632
chore: Polish comments for stat and stat_with by @Xuanwo in #3657
chore: clearer doc for python binding by @wcy-fdu in #3667
chore: Bump to v0.43.0 to start release process by @G-XD in #3672
chore: Bump to v0.43.0 to start release process (Round 2) by @G-XD in #3676
chore: add license.workspace to help cargo deny reports by @tisonkun in #3679
chore: clearer doc for list metakey by @wcy-fdu in #3666
chore: Bump to v0.43.0 to start release process (Round 3) by @G-XD in #3685

New Contributors

@Xuxiaotuan made their first contribution in #3543
@cjj2010 made their first contribution in #3546
@shbhmrzd made their first contribution in #3559
@AlexVCS made their first contribution in #3579
@jtescher made their first contribution in #3580
@JackDrogon made their first contribution in #3581
@qingwen220 made their first contribution in #3592
@ZutJoe made their first contribution in #3652
@vincentdephily made their first contribution in #3683

Full Changelog: v0.42.0...v0.43.0