Arweave 2.7.4 Release Notes

If you were previously running the 2.7.4 pre-release we recommend you update to this release. This release includes all changes from the pre-release, plus some additional fixes and features.

Mining Performance Improvements

This release includes a number of mining performance improvements, and is the first release for which we've seen a single-node miner successfully mine a full replica at almost the full expected hashrate (56 partitions mined at 95% efficiency at the time of the test). If your miner previously saw a loss of hashrate at higher partition counts despite low CPU utilization, it might be worth retesting.

Erlang VM arguments

Adjusting the arguments provided to the Erlang VM can sometimes improve mining hashrate. In particular we found that on some high-core count CPUs, restricting the number of threads available to Erlang actually improved performance. You'll want to test these options for yourself as behavior varies dramatically from system to system.

This release introduces a new command-line separator: --

All arguments before the -- separator are passed to the Erlang VM, all arguments after it are passed to Arweave. If the -- is omitted, all arguments are passed to Arweave.

For example, to restrict the number of threads available to Arweave to 24, you would build a command like:

./bin/start +S 24:24 -- <regular arweave command line flags>

Faster Node Shutdown

Unrelated to the above changes, this release includes a couple fixes that should reduce the time it takes for a node to shut down following the ./bin/stop command.

Solution recovery

This release includes several features and bug fixes intended to increase the chance that a valid solution results in a confirmed block.

Rebasing

When two or more miners post blocks at the same height, the block that is adopted by a majority of the network first will be added to the blockchain and the other blocks will be orphaned. Miners of orphaned blocks do not receive block rewards for those blocks.

This release introduce the ability for orphaned blocks to be rebased. If a miner detects that their block has been orphaned, but the block solution is still valid, the miner will take that solution and build a new block with it. When a block is rebased a rebasing_block message will be printed to the logs.

Last minute proof fetching

After finding a valid solution a miner goes through several steps as they build a block. One of those steps involves loading the selected chunk proofs from disk. Occasionally those proofs might be missing or corrupt. Prior to this release when that happened, the solution would be rejected and the miner would return to hashing. With this release the miner will reach out to several peers and request the missing proofs - if successful the miner can continue building and publishing the block.

last_step_checkpoints recovery

This release provides more robust logic for generating the last_step_checkpoints field in mined blocks. Prior to this release there were some scenarios where a miner would unnecessarily reject a solution due to missing last_step_checkpoints.

VDF Server Improvements

In addition to a number of VDF server/client bug fixes and performance improvements, this release includes two new VDF server configurations.

VDF Forwarding

You can now set up a node as a VDF forwarder. If a node specifies both the vdf_server_trusted_peer and vdf_client_peer flags it will receive its VDF from the specified VDF Servers and provide its VDF to the specified VDF clients. The push/pull behavior remains unchanged - any of the server/client relationships can be configured to push VDF updates or pull them

Public VDF

If a VDF server specifies the enable public_vdf_server flag it will provide VDF to any peer that requests it without needing to first whitelist that peer via the vdf_client_peer flag.

/recent endpoint

This release adds a new /recent endpoint which will return a list of recent forks that the node has detected, as well as the last 18 blocks they've received as well as the timestamps they received them.

Webhooks

This release adds additional webhook support. When webhooks are configured a node will POST data to a provided URL (aka webhook) when certain events are triggered.

Node webhooks can only be configured via a JSON config_file. For example:

{
  "webhooks": [
    {
      "events": ["transaction", "block"],
      "url": "https://example.com/block_or_tx",
      "headers": {
        "Authorization": "Bearer 123"
       }
    },
    {
      "events": ["transaction_data"],
      "url": "http://127.0.0.1:1985/tx_data"
    }
}

The supported events are:

• transaction : POSTS
  • the transaction header whenever this node accepts and validates a new transaction
• transaction_data : POSTS
  • { "event": "transaction_data_synced", "txid": <TXID> } once this node has received all the chunks belonging to the transaction TXID
  • { "event": "transaction_orphaned", "txid": <TXID> } when this node detects that TXID has been orphaned
  • { "event": "transaction_data_removed", "txid": <TXID> } when this node detects that at least one chunk has been removed from a previously synced transaction
• block : POSTS
  • the block header whenever this node accepts and validates a new block

In all cases the POST payload is JSON-encoded

Benchmarking and data utilities

• ./bin/benchmark-hash prints benchmark data on H0 and H1/H2 hashing performance
• fix for ./bin/data-doctor bench - it should now be able to correctly report storage module read performance
• data-doctor dump dumps all block headers and transactions

Miscellaneous Bug Fixes and additions

1. Several coordinated mining and mining pool bug fixes
2. /metrics was incorrect if mining address included a _
3. Fix bug in start_from_block and start_from_latest_state
4. Add CORS header to /metrics so it can be queried from an in-browser app
5. Blacklist handling optimizations