This is an alpha update and may not be ready for production use. This software was prepared by the Digital History Association, in cooperation from the wider Arweave ecosystem.
This release includes several bug fixes. It passes all automated tests and has undergone a base level of internal testing, but is not considered production ready. We only recommend upgrading if you believe one of the listed bug fixes will improve your mining experience.
verify Tool Improvements
This release contains several improvements to the verify tool. Several miners have reported block failures due to invalid or missing chunks. The hope is that the verify tool improvements in this release will either allow those errors to be healed, or provide more information about the issue.
New verify modes
The verify tool can now be launched in log or purge modes. In log mode the tool will log errors but will not flag the chunks for healing. In purge mode all bad chunks will be marked as invalid and flagged to be resynced and repacked.
To launch in log mode specify the verify log flag. To launch in purge mode specify the verify purge flag. Note: verify true is no longer valid and will print an error on launch.
Chunk sampling
The verify tool will now sample 1,000 chunks and do a full unpack and validation of the chunk. This sampling mode is intended to give a statistical measure of how much data might be corrupt. To change the number of chunks sampled you can use the the verify_samples option. E.g. verify_samples 500 will have the node sample 500 chunks.
More invalid scenarios tested
This latest version of the verify tool detects several new types of bad data. The first time you run the verify tool we recommend launching it in log mode and running it on a single partition. This should avoid any surprises due to the more aggressive detection logic. If the results are as you expect, then you can relaunch in purge mode to clean up any bad data. In particular, if you've misnamed your storage_module the verify tool will invalidate all chunks and force a full repack - running in log mode first will allow you to catch this error and rename your storage_module before purging all data.
Bug Fixes
- Fix several issues which could cause a node to "desync". Desyncing occurs when a node gets stuck at one block height and stops advancing.
- Reduce the volume of unnecessary network traffic due to a flood of
404requests when trying to sync chunks from a node which only servesreplica.2.9data. Note: the benefit of this change will only be seen when most of the nodes in the network upgrade. - Performance improvements to HTTP handling that should improve performance more generally.
- Add TX polling so that a node will pull missing transactions in addition to receiving them via gossip
Community involvement
A huge thank you to all the Mining community members who contributed to this release by identifying and investigating bugs, sharing debug logs and node metrics, and providing guidance on performance tuning!
Discord users (alphabetical order):
- AraAraTime
- BerryCZ
- bigbang
- BloodHunter
- Butcher_
- dlmx
- dzeto
- edzo
- EvM
- Fox Malder
- Iba Shinu
- JF
- jimmyjoe7768
- lawso2517
- MaSTeRMinD
- MCB
- Methistos
- Michael | Artifact
- qq87237850
- Qwinn
- RedMOoN
- smash
- sumimi
- T777
- Thaseus
- Vidiot
- Wednesday
- wybiacx
What's Changed
Full Changelog: N.2.9.4...N.2.9.5-alpha1