Arweave 2.9.1

This Arweave node implementation proposes a hard fork that activates at height 1602350, approximately 2025-02-03 14:00 UTC. This software was prepared by the Digital History Association, in cooperation with the wider Arweave ecosystem. Additionally, this release was audited by NCC Group.

Note: with 2.9.1 when enabling the randomx_large_pages option you will need to configure 5,000 HugePages rather than the 3,500 required for earlier releases.

Replica 2.9 Format

The primary focus of this release is to complete the implementation, validation, and testing of the Replica 2.9 Format introduced in the previous "early adopter" release: 2.9.0-early-adopter. Those release notes are still a good source of information about the Replica 2.9 Format.

With this 2.9.1 release the Replica 2.9 Format is ready for production use. New and existing miners should consider packing or repacking to the replica.2.9 format.

Note: If you have replica.2.9 data that was previously packed with the 2.9.0-early-adopter release, please delete it before running 2.9.1. There are changes in 2.9.1 which render it incompatible with previously packed replica.2.9 data. spora_2_6 and composite data is unaffected.

Benefits of the Replica 2.9 Format

Arweave 2.9’s format enables:

• Allow miners to read from their drives at a rate of 5 MiB/s (the equivalent of difficulty 10 in Arweave 2.8), without adversely affecting the security of the network. This represents a decrease of 90% from Arweave 2.8, and 97.5% from Arweave 2.7.x. This will allow miners to use the most cost efficient drives in order to participate in the network, while also lowering pressure on disk I/O during mining operations.
• A ~96.9% decrease in the compute necessary to pack Arweave data when compared to 2.8 composite.1, and SPoRA_2.6. This decrease also scales approximately linearly for higher packing difficulties. For example, for miners that would have packed with Arweave 2.8 to the difficulty necessary to reach a 5 MB/s read speed (composite.10), Arweave 2.9 will require ~99.56% less energy and time. This represents an efficiency improvement of 32x against 2.7.x and 2.8 composite.1, and ~229x for composite.10.

Packing Performance

Arweave packing consists of two phases:

1. Entropy generation
2. Chunk enciphering

In prior packing formats (e.g. spora_2_6 and composite) those phases were merged: for each chunk a small bit of entropy was generated and then the chunk was enciphered. Historically the entropy generation has been the bottleneck and main driver of CPU usage.

With replica.2.9 the phases are separated. Entropy is generated for many chunks, and then that entropy is read and many chunks are enciphered. The entropy generation phase is many times faster than it was for spora_2_6 and composite - in our benchmarks a single node is able to generate entropy for the full weave in ~3 days. The CPU requirements for the enciphering phase are also quite low as enciphering is now a lightweight XOR operation. The end result is that now disk IO is the main bottleneck when packing to replica.2.9.

We have updated the docs to provide guidance on how to approach repacking to replica.2.9: Syncing and Packing Guide

We are working on a follow-up release which will attempt to further optimize the disk IO phase of the packing process.

Changes from the 2.9.0-early-adopter release

• Previously there was a limitation which degraded packing performance for non-contiguous storage modules. This has been addressed. You can now pack singular and non-contiguous storage modules with no impact on packing performance.
• All modes of packing to replica.2.9 are supported. I.e. "sync and pack", "cross-module repack", and "repack-in-place" are all supported. However you are not yet be able to repack from replica.2.9 to any other format.
• Overall packing performance has improved. Further work is needed to streamline the disk IO during the packing process.
• The packing_rate flag is now deprecated and will have no impact. It's been replaced by the packing_workers flag which allows you to set how many concurrent worker threads are used while packing. The default is the number of logical cores in the system.
• The replica_2_9_workers flag controls how many storage modules the node will generate entropy for at once. Only one storage module per physical device will have entropy generated at a time. The default is 8, but the optimal value will vary from system to system.
• We've update the Metrics Guide with a new Syncing and Packing Grafana dashboard to better visualize the replica.2.9 packing process.

Support for ECDSA Keys

This release introduces support for ECDSA signing keys. Blocks and transactions now support ECDSA signatures and can be signed with ECDSA keys. RSA keys continue to be supported and remain the default key type.

An upcoming arweave-js release will provide more guidance on using ECDSA keys with the Arweave network.

ECDSA support will activate at the 2.9 hard fork (block height 1602350).

Composite Packing Format Deprecated

The new packing format was discovered as a result of researching an issue (not endangering data, tokens, or consensus) that affects higher difficulty packs of the 2.8 composite scheme. Given this, and the availability of the significantly improved 2.9 packing format, as of block height 1642850 (roughly 2025-04-04 14:00 UTC), data packed to any level of the composite packing format will not produce valid block solutions.

What's Changed

Full Changelog: N.2.8.3...N.2.9.1