Summary

This is a major release with new features and lots of bug fixes.

General

• GPU-Aware I/O enabled by using Kokkos. Device pointers can be passed to Put()/Get() calls directly. Kokkos 3.7.x required for this release. Works with CUDA, HIP and Kokkos applications. https://adios2.readthedocs.io/en/latest/advanced/gpu_aware.html#gpu-aware-i-o
• GPU-compression. MGARD and ZFP operators can compress data on GPU if they are built for GPU. MGARD operator can be fed with host/device pointers and will move data automaticaly. ZFP operator requires matching data and compressor location.
• Joined Array concept (besides Global Array and Local Array), which lets writers dump Local Arrays (no offsets no global shape) that are put together into a Global Array by the reader. One dimension of the arrays is selected for this join operation, while other dimensions must be the same for all writers. https://adios2.readthedocs.io/en/latest/components/components.html?highlight=Joined#shapes

File I/O

• Default File engine is now BP5. If for some reason this causes problems, manually specify using "BP4" for your application.
• BP5 engine supports multithreaded reading to accelerate read performance for low-core counts.
• BP5 Two level metadata aggregation and reduction reduced memory impact of collecting metadata and therefore is more scalable in terms of numbers of variables and writers than BP4.
• Uses Blosc-2 instead of Blosc for lossless compression. The new compression operator is backward compatible with old files compressed with blosc. The name of the operator remains "blosc".

Staging

• UCX dataplane added for SST staging engine to support networks under the UCX consortium
• MPI dataplane added for SST staging engine. It relies on MPI intercommunicators to connect multiple independent MPI applications for staging purposes. Applications must enable multithreaded MPI for this dataplane.

Experimental features

• Preliminary support for data structs. A struct can have single variables of basic types, and 1D fixed size arrays of basic types. Supported by BP5, SST and SSC engines.

What's Changed

• Refactor SSC to allow easy addition of new sub-routines by @JasonRuonanWang in #3092
• BP5Deserializer GenerateReadRequests fix rank fix by @eisenhauer in #3096
• Added naive engine mode for SSC by @JasonRuonanWang in #3109
• move sodium option to its right place by @JasonRuonanWang in #3112
• Bump pillow from 9.0.0 to 9.0.1 in /docs by @dependabot in #3103
• added bp file reading in MgardPlus by @JasonRuonanWang in #3113
• added zero block test for ssc by @JasonRuonanWang in #3111
• fixed a bug in ssc which blocks MPI communications by @JasonRuonanWang in #3114
• clean up SSC logging information for better debugging experience by @JasonRuonanWang in #3115
• SSC: temporary workaround for crusher MPI bug by @JasonRuonanWang in #3117
• DILL upstream by @eisenhauer in #3118
• Fix windows mpi builds by @chuckatkins in #3142
• remove template instantiation in header files by @JasonRuonanWang in #3144
• MgardPlus: check for empty mesh file by @JasonRuonanWang in #3157
• pr3153 into master by @chuckatkins in #3158
• blosc unknown parameter should be warning by @JasonRuonanWang in #3163
• Fix a template instantiation issue with GCC 10 in dataman serializer by @JasonRuonanWang in #3165
• ENET upstream by @eisenhauer in #3166
• Fix helloBPTimeWriter Python example. by @manauref in #3167
• removed VariableCompound class as it has never worked since added by @JasonRuonanWang in #3168
• move Span class to dedicated files by @JasonRuonanWang in #3169
• Add template-free C++ API by @JasonRuonanWang in #3171
• on-demand step delivery by @eisenhauer in #3170
• added missing open modes in C API by @JasonRuonanWang in #3174
• Add struct variable support by @JasonRuonanWang in #3179
• added DefineStructVariable test by @JasonRuonanWang in #3183
• moved Variable::Shape() to VariableBase by @JasonRuonanWang in #3184
• removed VariableBase::GetShape by @JasonRuonanWang in #3185
• Remove templates in SSC writer by @JasonRuonanWang in #3187
• removed most templated functions in ssc reader by @JasonRuonanWang in #3188
• added ssc test for struct variables by @JasonRuonanWang in #3189
• added IO::m_StructDefinitions to hold temporary struct definitions from parsing step metadata by @JasonRuonanWang in #3191
• fixed a bug in ssc test that fails when mpi size is large by @JasonRuonanWang in #3192
• Add MinMax in non-template API by @JasonRuonanWang in #3193
• Tweak OnDemand test by @eisenhauer in #3190
• Add BlocksInfo in SSC naive by @JasonRuonanWang in #3194
• finish up polishing tests and examples by @JasonRuonanWang in #3196
• moved ToBlocksInfoMin to private by @JasonRuonanWang in #3197
• added BlocksInfo API for VariableNT, and implementation for VariableStruct in SSC by @JasonRuonanWang in #3198
• make struct size parameter compulsory to ensure padding correctness by @JasonRuonanWang in #3199
• added freeze function to StructDefinition by @JasonRuonanWang in #3200
• added allowReorganize for InquireStructVariable by @JasonRuonanWang in #3201
• Add SSC struct definition serialization by @JasonRuonanWang in #3202
• Update documentation with correct StepDistributionMode values by @anagainaru in #3212
• Add access to internal m_BetweenStepPairs member by @eisenhauer in #3216
• Bp5 read multithreaded by @pnorbert in #3220
• Bp5 multithreaded read, dynamic version by @pnorbert in #3233
• Throw an error when GathervArrays() is asked to gather more than 2^31… by @pnorbert in #3239
• Stop using Dims in some performance critical areas by @eisenhauer in #3240
• Merge bp5 flush read from 281 by @pnorbert in #3243
• Fix cuda build by @eisenhauer in #3246
• If BP5 parameter Threads is 0 (default), calculate the number of thre… by @pnorbert in #3247
• fix api for gcc11 by @robertu94 in #3250
• BP5 Two level metadata aggregation and reduction: work in progress saved by @pnorbert in #3251
• Remove DebugMode from all external APIs by @eisenhauer in #3204
• FFS upstream by @eisenhauer in #3259
• Fix C binding problem by @eisenhauer in #3264
• BP5 subformats by @eisenhauer in #3260
• Ensure Init of Blocks Info by @eisenhauer in #3268
• Kill reader-created variables in parent IO upon engine close (for BP5) by @eisenhauer in #3270
• Changed parameters for DAOS API functions. by @dmitry-ganyushin in #3266
• Add backward compatible read with BLOSC compression to BP4 files crea… by @pnorbert in #3277
• Necessary revision to get VOL work HDF5 1.13 & BP4 by @guj in #3279
• Destructor close by @eisenhauer in #3278
• Destroy only created vars on BeginStep and again on Reader close by @eisenhauer in #3272
• VOL: added BP5 read support for H5 style files (no steps) by @guj in #3282
• Bump version to v2.8.2 on master by @chuckatkins in #3281
• Fix a typo. by @hyoklee in #3283
• fix bpls: if minblockinfo is available, don't just still use allsteps… by @pnorbert in #3288
• CI: adds CUDA build by @vicentebolea in #3276
• Fix bpls dumping local arrays from BP5 files. Also renamed WasLocalVa… by @pnorbert in #3289
• SST: Add MPI SST dataplane by @vicentebolea in #3095
• MPI_DP: Optimize scalable component by @vicentebolea in #3292
• Optimize processing attributes, by using a new AttributeBase.Equals()… by @pnorbert in #3299
• Span MinMax in BP5 by @eisenhauer in #3306
• ci: fix macos install test missing deps by @vicentebolea in #3308
• Enable memory selection in ssc reader by @JasonRuonanWang in #3313
• zfp: fix version compatibility for 1.0+ by @chuckatkins in #3312
• Rework BP5 Attribute handling by @eisenhauer in #3304
• fix compile error in dataspace engine by @JasonRuonanWang in #3317
• When reading with different order (column <-> row order), do not reve… by @pnorbert in #3315
• Fix MinInfo Leak, add tests by @eisenhauer in #3316
• TESTS, Examples: Add comment for MPI_THREAD_MULTIPLE by @vicentebolea in #3293
• Modifiable attributes for staging and BP5 by @eisenhauer in #3318
• cmake: make find_package(ZFP) quiet by @vicentebolea in #3319
• Fix compilation problem on crusher by @eisenhauer in #3321
• SST with MPI dataplane: only print MPI initialization warning on rank 0 by @cwsmith in #3322
• Reinstall Norbert's BP5Deserializer opts by @eisenhauer in #3327
• CI: update macOS version for github OS builds by @vicentebolea in #3325
• utils: Fix bpls linking in Windows by @Biswa96 in #3242
• Restore inadverent changes to Attr handling by @eisenhauer in #3333
• CUDA: remove cuda_drivers linking dep by @vicentebolea in #3334
• CI: major rework for image generations of GHA by @vicentebolea in #3331
• CI: add cuda_lambda to Kokkos deployment by @vicentebolea in #3337
• AdiosView object that allows Get/Put to receive Kokkos::View by @anagainaru in #3320
• CMAKE: CMAKE_CXX_EXTENSIONS=OFF by @vicentebolea in #3341
• CMAKE: Use Ninja for gcc[9,10,11] by @vicentebolea in #3338
• CI: provide catalyst to gcc11 by @vicentebolea in #3340
• Untemplate BP5Writer::PutCommon by @eisenhauer in #3343
• Handshake fix: rank 0 should receive msg from last rank, not from -1. by @pnorbert in #3345
• Support Structs in BP5 marshalling (BP5file and SST), modify an SSC t… by @eisenhauer in #3350
• CMAKE: Support cmake 3.24 by @vicentebolea in #3351
• Fix race condition in OnDemand timestep distribution by @eisenhauer in #3355
• EVpath upstream updates by @eisenhauer in #3360
• SST reader tries to use the writer's DataPlane or fail by @eisenhauer in #3356
• Handle error on SST contact file create by @eisenhauer in #3364
• Fix race, restore multi-reader OnDemand test by @eisenhauer in #3361
• ParaView Catalyst In Situ engine plugin by @caitlinross in #3346
• Fix race condition in OnDemand delivery by @eisenhauer in #3369
• Added BLOSC2 operator using the c-blosc-2 library. It works identical… by @pnorbert in #3374
• Remove closed file from map of transports so that we don't attempt to close it again by @pnorbert in #3375
• Raise the limit of open filesi to max in BP file engines by @pnorbert in #3376
• Clean-up by @dmitry-ganyushin in #3380
• repeat setup from Init in Execute (Caitlin's fix) by @pnorbert in #3384
• Fix conversion warning in CompressPNG.cpp by @eisenhauer in #3389
• BP5 read direct to application memory (1 dimensional case) by @eisenhauer in #3387
• WIP: Fix untyped conversion in PNG operator test by @eisenhauer in #3390
• CI: Workaround for Windows builds 20221120.1 by @vicentebolea in #3394
• Convert Transport parameters's keys to lower case for easier lookup. by @pnorbert in #3388
• Add new parameters to bpls to control engine, engine params and trans… by @pnorbert in #3399
• Change (at least) some sprintf() to snprintf() by @eisenhauer in #3401
• Added a simple example for Fides: a single hexagon cell output by @pnorbert in #3404
• Add Python SstWriter example by @eisenhauer in #3403
• Revert "CI: Workaround for Windows builds 20221120.1" by @vicentebolea in #3405
• Move Ssc-internal struct manipulation to Ssc-Internal Maps by @eisenhauer in #3406
• Rename some portions of Struct interface by @eisenhauer in #3409
• GPU buffers will never bypass the ADIOS internal buffers by @anagainaru in #3410
• Adjust DefineStruct bindings by @eisenhauer in #3414
• install: post install test checks for jail root builds by @vicentebolea in #3415
• Clean the GPU code to allow multiple backends by @anagainaru in #3411
• CMAKE: Check usability of MPI_Port_open by @vicentebolea in #3407
• Dill upstream by @eisenhauer in #3420
• Testing for GPU backend for correct/incorrect memory spaces by @anagainaru in #3421
• Allowing the option of CUDA memory space only if CUDA is enabled by @anagainaru in #3423
• SST: Add UCX SST Dataplane by @sameehj in #3416
• Added the AWSSDK transport using the AWS SDK S3 API. by @pnorbert in #3425
• When we use AWSSDK, bpls should not check if the file it exists loca… by @pnorbert in #3431
• Enforce minimum UCX version by @eisenhauer in #3432
• Kill warnings in UCX dataplane by @eisenhauer in #3435
• Extend FindUCX to work if pkgconfig is missing or ignored by @eisenhauer in #3437
• Fix MGARD operator by @JieyangChen7 in #3438
• Blosc by @lizdulac in #3430
• Do not use the deprecated ctime(), use instead localtime_r() and strf… by @pnorbert in #3441
• Fix BP5 Append: fix metadata index position calculation for AppendAft… by @pnorbert in #3447
• sst,mpi: renable ADIOS2_SST_HAVE_MPI by @vicentebolea in #3449
• Fig bug in the IME file transport by @anagainaru in #3451
• Reader-side struct API changes by @eisenhauer in #3428
• Testing for MGARD operator with GPU buffers by @anagainaru in #3434
• Tweak min blocks interface to avoid leak in Shape() by @eisenhauer in #3458
• Fix UCX version checking by @eisenhauer in #3464
• Added support for Joined Arrays in the BP4 format and engine. by @pnorbert in #3466
• Warning fix when finding the MGARD library by @anagainaru in #3467
• Fix bug in BP3/BP4 attribute merging process. When a global attribute… by @pnorbert in #3468
• Add JoinedArray test in staging-common, limit to BP4 currently by @eisenhauer in #3469
• Bpls daos extention by @dmitry-ganyushin in #3474
• Fixed: When appending to BP5 with a different number of processes, bp… by @pnorbert in #3472
• Better error when libfabric defaults to or is specified a bad FABRIC_IFACE by @eisenhauer in #3475
• release: Bump version to v2.9.0-rc1 by @vicentebolea in #3476
• Fix misplaced *s in sst.rst by @tobyjamez in #3471
• Fix build error on Summit for PGI with CUDA backend by @anagainaru in #3481
• Make adios_iotest work with SST again: add CurrentStep() to SstWriter… by @pnorbert in #3482
• Fix Matlab build which depends on the C bindings not the C++ bindings. by @pnorbert in #3489
• BP5 JoinedArray support by @eisenhauer in #3488
• Add JoinedArrays to SST in BP5 marshaling by @eisenhauer in #3491
• Kokkos backend in ADIOS2 by @anagainaru in #3446
• Update the documentation for the GPU backend by @anagainaru in #3495
• kokkos,cmake: remove kokkosview example by @vicentebolea in #3498
• Remove deprecated bindings for release by @eisenhauer in #3494
• ci,kokkos,cuda: remove warnings by @vicentebolea in #3500
• cmake: do not install empty sst dirs by @vicentebolea in #3501
• Workaround for lammps and Scorpion to have a green CI by @vicentebolea in #3502
• Update fortran.rst by @mathrack in #3506
• Change default marshalling for SST to BP5 by @eisenhauer in #3513
• Set the CMAKE GPU architectures to what is used for Kokkos by @anagainaru in #3517
• Move the buffer copy logic outside operators by @anagainaru in #3514
• Add adios2_mode_readRandomAccess to fortran bindings by @eisenhauer in #3522
• Fix plugin engine example by @caitlinross in #3499
• update docs/README.md for using conda env by @caitlinross in #3427
• Dependbot backports by @vicentebolea in #3536
• Bug fix in BP5 when memory selection is used with GPU buffers by @anagainaru in #3539
• Fix bug when writing null blocks by @anagainaru in #3542
• Throw an exception in Python bindings on Get() buffer type mismatch by @eisenhauer in #3544
• Throw an exception if an unimplemented function is called in BP5 by @eisenhauer in #3543
• Fix for the issue 3503. by @dmitry-ganyushin in #3519
• H5 subfile support by @guj in #3530
• Refactoring and clean-up group API by @dmitry-ganyushin in #3557
• Change default file engine to BP5 by @eisenhauer in #3526
• I3454 missing mpi functions in doc by @dmitry-ganyushin in #3558
• Add documentation for Joined Arrays by @eisenhauer in #3554
• Collect Shape at EndStep (and restore DataWrite test) by @eisenhauer in #3562
• Bug fix for the Kokkos backend used with a Kokkos application by @anagainaru in #3560
• Throw an error when unsupported parameters are used with the ZFP CUDA backend by @anagainaru in #3567
• Bump version to v2.9.0 by @vicentebolea in #3568

New Contributors

• @manauref made their first contribution in #3167
• @robertu94 made their first contribution in #3250
• @hyoklee made their first contribution in #3283
• @cwsmith made their first contribution in #3322
• @Biswa96 made their first contribution in #3242
• @sameehj made their first contribution in #3416
• @lizdulac made their first contribution in #3430
• @tobyjamez made their first contribution in #3471
• @mathrack made their first contribution in #3506

Full Changelog: v2.8.0...v2.9.0