Release notes
C API Improvements
New two-call pattern functions for safely retrieving string data of unknown
length: adios2_get_string(), adios2_attribute_string_data(), and
adios2_attribute_string_data_array().
Selection API for Get() Operations
A new Selection class bundles spatial, block, and step parameters into a
single object passed to Get(), replacing scattered SetSelection() /
SetBlockSelection() / SetStepSelection() calls on Variable.
engine.Get(var, data, adios2::Selection::BoundingBox({0,0},{10,20}).WithSteps(0,5));Plugin Interface v2 (Breaking Change)
Engine and operator plugins no longer inherit from internal core types.
Engine plugins inherit from PluginEngineInterface, operator plugins from
PluginOperatorInterface, and shared libraries only need to link adios2::cxx.
Update EngineCreate() / EngineDestroy() signatures; see examples/plugins/.
C++17 Minimum Requirement
The minimum C++ standard to build ADIOS2 has been raised from C++14 to C++17.
S3 Object Storage for BP5 Data
BP5 can now write data files to S3-compatible object storage (Amazon S3, MinIO,
Ceph, etc.) with metadata kept on the local filesystem. Set
DataFileTransport=awssdk, S3Endpoint, and S3Bucket as engine parameters.
Python Bindings: Nanobind Migration
The Python bindings have been migrated from pybind11 to nanobind, producing
smaller, faster extension modules and enabling stable-ABI (abi3) wheels for
Python 3.12 and later.
Asymmetric Encryption Operator
A new Encryption operator encrypts variable data using libsodium's sealed-box
construction (X25519 + XSalsa20-Poly1305). Writers only need the recipient's
public key; only the private key holder can decrypt.
SZ3 Compression Operator
A new SZ3 lossy compression operator is available, extending the existing
options alongside SZ, ZFP, MGARD, and BigWhoop.
RefactorProDM Operator
A new RefactorProDM operator integrates ADIOS2 with ProDM (Dr. Xin Liang's
decomposition library), providing decomposition-based data reduction when ProDM
is available at build time.
Python Free-Threading Support
The Python bindings are now compatible with free-threaded Python (PEP 703,
python3.13t), allowing ADIOS2 to run without the GIL.
HTTPS Transport for Remote Access
An HTTPS-based remote transport (XrootdHttpsRemote) using libcurl has been
added as an alternative to the native XRootD protocol for environments where
native XRootD connectivity is unavailable.
XRootD Server Resource Management
The XRootD server plugin now evicts idle file descriptors and metadata when
configurable limits are approached, shares a single FD across engines reading
the same tar file, and batches variable reads into a single network round-trip.
Cross-Endian Interoperability
The ADIOS2_USE_Endian_Reverse CMake option has been removed; cross-endian
file interoperability is now always enabled.
Campaign Reader Improvements
Host aliases, XRootD host entries, and in-memory metadata caching were added.
The reader now requires ACA 0.7 strictly. A rewritten SQLite join query reduces
open time from ~27 seconds to under one second for large archives.
TAR File Reading
BP5 datasets, text files, and images can now be read from TAR archives (local,
HTTPS-hosted, or via the remote server) when a TAR index is provided.
BP5 DataFileTransport Parameter Rename
The BP5 engine parameter DataTransport has been renamed to DataFileTransport.
Existing configurations must be updated.
BP5 Concurrent File Access
BP5 data reading now uses per-subpool locking over a shared file-descriptor
pool, enabling safe concurrent reads from multiple threads.
Python Stream: minmax()
adios2.Stream.minmax(name, [step], [block_info_list]) returns per-step or
global min/max values of a variable.
Inline Engine Buffering
The inline engine now supports basic output buffering, moving toward feature
parity with file-based engines for in-situ workflows.
Windows and macOS pip Wheel Support
CI now runs pip sdist and wheel integration tests on Windows and macOS,
verifying correct library loading under pip install workflows.
REUSE License Compliance
ADIOS2 now complies with the REUSE specification; every source file carries a
machine-readable SPDX license and copyright notice.
What's Changed
- Fix logic error in BP5 selective metadata aggregation by @eisenhauer in #4717
- RDMA dataplane: Share fi_fabric() and don't close until all are done by @eisenhauer in #4646
- Potential fix for CMake logic in using an external EVPath build by @Crivella in #4697
- backport 4722 by @vicentebolea in #4724
- Bump the all-actions group with 3 updates by @dependabot[bot] in #4727
- cmake: define cxx11 target once by @vicentebolea in #4730
- Korvo Upstream by @eisenhauer in #4733
- Fixup XRootD plugin by @eisenhauer in #4728
- ci: fix windows ci python err by @vicentebolea in #4739
- ci: upload conda env description to cdash by @vicentebolea in #4744
- use the same base handle as in extract.sh by @guj in #4714
- Use Transports directly by @eisenhauer in #4740
- ci: add backport action by @vicentebolea in #4748
- ci: setup git committer in backport action by @vicentebolea in #4749
- ci: setup git committer in backport action by @vicentebolea in #4750
- ci: setup git committer in backport action by @vicentebolea in #4751
- Note SZ version operability by @eisenhauer in #4752
- BP5Writer: Subfile config for DSB works like other aggregators by @scottwittenburg in #4742
- Documentation fixes for beginner users by @hahnjo in #4746
- Better documentation of collective functions by @hahnjo in #4755
- ci: remove temp install after test install by @vicentebolea in #4754
- ci: download latest cmake for el8 imgs by @vicentebolea in #4758
- Updated profiler tags used in json by @guj in #4759
- Fix Conda badge link in ReadMe.md by @vicentebolea in #4762
- Add support to read a BP5 dataset / TEXT / IMAGE from TAR files... by @pnorbert in #4764
- Use specified engine rather than BPFile in tests by @eisenhauer in #4757
- Integrate ProDM changes, added new operator RefactorProDM by @AkiraFukuda in #4771
- Add SZ3 compression operator support by @eisenhauer in #4772
- Use pool of files for data reading in BP5 by @eisenhauer in #4766
- Add CLAUDE.md guidance file for Claude Code development by @eisenhauer in #4779
- Add automatic cleanup of generated test files in BP engine tests by @eisenhauer in #4778
- Consolidate test utilities and improve cleanup handling by @eisenhauer in #4780
- Order remote replicas in order of most likely accessible: … by @pnorbert in #4768
- Fix SZ3 3.1+ API compatibility in decompression by @eisenhauer in #4785
- ci: add EXTRA_SPECS build arg to spack imgs by @vicentebolea in #4786
- ci: use 0s in static jobs by @vicentebolea in #4788
- ci: fix clang static job by @vicentebolea in #4792
- tests: sanitize path before removal by @vicentebolea in #4789
- Update common tracking by @mathstuf in #4710
- build(deps): bump the all-actions group across 1 directory with 7 updates by @dependabot[bot] in #4787
- tests: sanitize path before removal by @vicentebolea in #4793
- refactor code to utilize c++14 features by @vicentebolea in #4783
- ci: reduce size of clang11-static-mpich job by @vicentebolea in #4798
- Add adios2_get_string() to C API for safe string variable retrieval by @eisenhauer in #4801
- Add C API functions for safe string attribute retrieval by @eisenhauer in #4804
- build(deps): bump github/codeql-action from 4.31.9 to 4.31.10 in the all-actions group by @dependabot[bot] in #4797
- Reorganize reading of ACA files for version 0.7 by @pnorbert in #4809
- build(deps): bump actions/cache from 5.0.1 to 5.0.2 in the all-actions group by @dependabot[bot] in #4810
- Revert "Merge pull request #4710 from mathstuf/update-common-tracking" by @vicentebolea in #4811
- GTKorvo Upstream by @eisenhauer in #4812
- Detect the new flag HDF5_PROVIDES_PARALLEL from 2.0 by @guj in #4814
- Add HTTPS transport for XRootD remote access by @eisenhauer in #4815
- adios2.Stream.minmax() by @pnorbert in #4818
- ci: fix readthedocs job by @vicentebolea in #4823
- Fix 3p update script by @mathstuf in #4816
- build(deps): bump the all-actions group across 1 directory with 3 updates by @dependabot[bot] in #4824
- Xrootd https remote by @eisenhauer in #4822
- Fix dill config for unsupported architectures by @eisenhauer in #4826
- Fix BP5 cross-endian support for big-endian architectures (s390x) by @eisenhauer in #4803
- fix thirdparties by @vicentebolea in #4827
- Stabilize CI tests by @eisenhauer in #4829
- build(deps): bump github/codeql-action from 4.32.0 to 4.32.1 in the all-actions group by @dependabot[bot] in #4832
- BP5: Fix segfault with DataSizeBased aggregation and async write by @scottwittenburg in #4834
- Cleanup XRootD Docker build by @eisenhauer in #4828
- Fix flaky tests: BPFortranToCppRead and TestOperator.py by @eisenhauer in #4835
- Fix DataMan Python tests: add RUN_SERIAL by @eisenhauer in #4837
- Merge WriteReadADIOS2 test variants by @eisenhauer in #4830
- Fix NULL pointer dereference in SST writer close by @eisenhauer in #4839
- Add accuracy parameters to XRootD HTTP protocol by @eisenhauer in #4840
- Support host aliases by @pnorbert in #4841
- BP5Deserializer: capture selection state at queue time for thread safety by @eisenhauer in #4842
- BP5Reader: close metadata FDs when writer is inactive by @eisenhauer in #4846
- Cleanup files that might be lingering from earlier output by @eisenhauer in #4845
- Campaign reader engine must call PerformGets in DoClose to enable mul… by @pnorbert in #4853
- ADIOSFilePool: per-SubPool locking for concurrent file access by @eisenhauer in #4847
- Eliminate blockdiag in favor of uploaded svg and png images. by @eisenhauer in #4861
- Korvo Upstream by @eisenhauer in #4852
- Fix XRootD HTTPS test startup race by @eisenhauer in #4855
- Fix SST writer close race and KillWriter test timeout by @eisenhauer in #4858
- Fix TimeSeries MPI/Serial test collision by @eisenhauer in #4856
- Fix wrong sizeof in ReturnMetadataInfo format registration by @eisenhauer in #4859
- Add Selection API for thread-safe Get() operations by @eisenhauer in #4843
- BP5Reader: Fix a possible issue when writer map changes between steps by @scottwittenburg in #4865
- FileStdio: Fix transport issue when opening in append mode by @scottwittenburg in #4864
- Implement and Document Selection-based Get() by @eisenhauer in #4866
- Create parent directory of temporary .ats file. by @pnorbert in #4867
- security: publish score cards result to github security dashboard by @vicentebolea in #4868
- ci: add dependabot coverage into our python deps by @vicentebolea in #4872
- Update assignees format in dependabot.yml by @vicentebolea in #4873
- Add support for version 2.11.x in SECURITY.md by @vicentebolea in #4875
- build(deps): bump the all-actions group across 1 directory with 2 updates by @dependabot[bot] in #4874
- Support xrootd connections in hosts.yaml, with xrootd/http/https tran… by @pnorbert in #4870
- Eliminate old QueueGet by @eisenhauer in #4869
- ci: add dependabot coverage into our conda deps by @vicentebolea in #4877
- Pick archive hosts with 'fs' before 's3', so that xrootd is preferred… by @pnorbert in #4876
- ci: correct dependabot coverage of our conda deps by @vicentebolea in #4878
- ci: only do security updates in conda,pip,docker by @vicentebolea in #4880
- remove thirdparty ci files by @vicentebolea in #4881
- [StepSecurity] Apply security best practices by @step-security-bot in #4882
- build(deps): bump github/codeql-action from 4.32.3 to 4.32.4 in the all-actions group by @dependabot[bot] in #4886
- Through-file streaming tweaks, add BP5 testing by @eisenhauer in #4885
- Plugin engine interface by @eisenhauer in #4863
- Catch error thrown by Campaign engine when engine.all_blocks_info() i… by @pnorbert in #4892
- Fix segfault on empty array attribute by @eisenhauer in #4893
- Pool curl request handles for connection reuse and batch requests. by @eisenhauer in #4888
- More complete plugin engine interface, document changes to all plugin interfaces, add whatsnew entry noting breaking change. by @eisenhauer in #4894
- Add S3 write support with hybrid storage for BP5 by @eisenhauer in #4831
- Better XRootD eviction and tar FD reuse by @eisenhauer in #4895
- Minor XRootD tweaks (time-based eviction, docs and comment fix) by @eisenhauer in #4897
- Refactor engine parameter parsing by @scottwittenburg in #4901
- remove spurious xrootd log message by @eisenhauer in #4900
- Gray-scott docs update by @eisenhauer in #4898
- ci: test fix sz3 by adding zstd by @vicentebolea in #4896
- Rename BP5 DataTransport parameter to DataFileTransport by @eisenhauer in #4904
- Campaign in memory metadata by @pnorbert in #4903
- ci: add wheel integration tests by @vicentebolea in #4906
- Automate backport via Copilot agent on
backport:releaselabel or PR description instruction by @Copilot in #4909 - correct typo backport by @vicentebolea in #4910
- correct typo backport by @vicentebolea in #4912
- more capable bp5dbg (prototype) by @eisenhauer in #4908
- migrate to nanobind by @vicentebolea in #4887
- backport: revert migration to copilot by @vicentebolea in #4917
- Rerouting aggregator by @scottwittenburg in #4647
- build(deps): bump the all-actions group with 3 updates by @dependabot[bot] in #4902
- Basic buffering in inline engine by @eisenhauer in #4922
- Process and save in-memory-metadata object if file is part of a time-… by @pnorbert in #4923
- Add Asymmetric encryption support EncryptionOperator by @vicentebolea in #4883
- cmake: bump to c++ standard to c++17 by @vicentebolea in #4907
- python: enable free threading python support by @vicentebolea in #4925
- python: add windows sdist pip test by @vicentebolea in #4924
- reuse: add REUSE compliance by @vicentebolea in #4928
- fix typo in dependabot config file by @vicentebolea in #4931
- Remove copyright and license details from ReadMe.md by @vicentebolea in #4934
- build(deps): bump the all-actions group with 5 updates by @dependabot[bot] in #4932
- python: replace flake8/pylint/black with ruff by @vicentebolea in #4930
- Add MPI barrier before test file cleanup to prevent race conditions by @eisenhauer in #4936
- Restructure OnDemand test for accuracy and reliability by @eisenhauer in #4937
- Add collective operations table for engines by @eisenhauer in #4940
- [StepSecurity] Apply security best practices by @step-security-bot in #4942
- Various HTTP/XROOTD changes by @eisenhauer in #4926
- Campaign Reader engine doc. Some fixes to other files. by @pnorbert in #4944
- build(deps): bump the all-actions group with 3 updates by @dependabot[bot] in #4946
- Embed doc images rather than host at imgur by @eisenhauer in #4945
- only accept ACA 0.7, throw error on older and newer ACA versions by @pnorbert in #4947
- Performance improvements in null-data single-node write case by @eisenhauer in #4950
- Defer hosts.yaml parsing until first access by @eisenhauer in #4951
- CI: add ubuntu/gcc12/mpich job using only external dependencies by @vicentebolea in #4949
- Remove GlobalServices, make Kokkos and PerfStubs lazy singletons by @eisenhauer in #4952
- try just specifying the full path for clang in the individual job by @pnorbert in #4954
- Rewrite reading the database using a large join select to process dat… by @pnorbert in #4956
- Use access/secret keys in AWS explicitly only if they are not empty, … by @pnorbert in #4955
- update campaign management example, fix some warnings by @pnorbert in #4957
- Restore ATOM_SERVER_HOST default for ADIOS by @eisenhauer in #4958
- Bump version to v2.12.0-rc1 by @vicentebolea in #4961
New Contributors
- @Crivella made their first contribution in #4697
- @hahnjo made their first contribution in #4746
- @AkiraFukuda made their first contribution in #4771
- @step-security-bot made their first contribution in #4882
Full Changelog: v2.11.0...v2.12.0-rc1