ArweaveTeam/arweave N.2.5.1.0

Release 2.5.1.0

on GitHub

The release introduces a rework of the mempool management and propagation, a small miner optimization, and improvements of the syncing process, especially end game syncing and syncing of a large number of disconnected intervals.

• Transactions are not rejected when the mempool limit is reached; instead, the least valuable transactions are dropped from the memory pool;
• picking transactions for a block is made faster;
• the peers for gossip are chosen from the entire network (some preference still goes to the better-rated peers) to avoid propagation issues when the rating network becomes too clustered; blocks are also sent to the trusted peers (specified on startup);
• the default number of peers transactions are gossiped to is reduced to speed up propagation;
• the node does not only search for confirmed transactions when searching for missing transactions for a block;
• freshly synced block and transaction headers are stored in RocksDB now;
• a bug is fixed where already known transactions would only be immediately ignored on repeated submission with 5 seconds since the previous submission. At the same time, the identifiers are not kept in memory indefinitely, preventing unnecessary albeit slow increase of the memory footprint;
• the headers are always stored in the disk cache (the size is controlled by the disk_cache_size_mb parameter) to make v1 transactions with data quickly available for download before their data is picked up from the disk pool and indexed for mining;
• the redundant signature validation during mempool filtering is removed;
• the miner is accelerated ~5-10% by removing one extra in-memory chunk copy per mined chunk;
• upgraded erlang-rocksdb to 1.7.0; RocksDB 6.25.3;
• made various improvements to the syncing process;
• fixed tracking of the disk_pool_chunks_count metric;
• improved the disk pool processing in the presence of many repeated data_roots throughout the weave.

API changes

• The node will reject a v2 transaction with the attached data if the node fails to reconstruct its data_root from the data. It does not pose consensus issues because 1) nodes strip data before gossip 2) the data field is ignored when a node fetches a missing transaction itself. The motivation is to reduce the likelihood of not uploading the data by mistake, e.g., when a new client library uses this way of uploading but does not split the data accordingly.

Upgrade instructions

N.2.5.1.0 comes with the prebuilt binaries for the Linux and Darwin x86_64 platforms. arweave-2.5.1.0.linux-x86_64.tar.gz works at least on Ubuntu 20 and CentOS 8. To run on Ubuntu 18, grab arweave-2.5.1.0.ubuntu18-x86_64.tar.gz. CentOS 7 is not supported.

Download the corresponding archive and unpack it:

tar -xzf arweave-2.5.1.0.linux-x86_64.tar.gz

It makes sense to unpack it inside a dedicated directory. You can always move this directory around, but the miner may not work if you move only some of the files. The weave data would, by default, be stored in this directory, too, but it can be overridden using the data_dir command-line argument.
The archive contents look like this:

bin/  data/  erts-10.3/  lib/  releases/

To run the miner, execute ./bin/start . See the mining guide for more details.

If your OS/platform architecture is not on the list, check README for how to build the miner from sources.

If you want to run the miner from the existing Git folder, upgrade Erlang to version 23 or 24 and execute the following command:

git fetch --all --tags && git checkout -f N.2.5.1.0

You can now run the miner using the arweave-server script.

Support

For more details on how to run a mining node, please see our mining guide - https://docs.arweave.org/info/mining/mining-guide. If you have any issues upgrading or would like to know more about the release, feel free to reach out to us in the #mining channel on Discord Arweave Dev Talk server (https://discord.gg/DjAFMJc) or email us at team@arweave.org.