github near/nearcore 2.0.0-rc.1
on GitHub

latest releases: crates-0.27.0, 2.3.0, 2.3.0-rc.4...
pre-release4 months ago
CODE_COLOR: CODE_YELLOW_TESTNET
RELEASE_VERSION: 2.0.0
PROTOCOL_UPGRADE: TRUE
DATABASE_UPGRADE: TRUE
SECURITY_UPGRADE: FALSE

Protocol Upgrade Voting

This release contains two protocol upgrades. The first protocol upgrade introduces the congestion control feature, and the second upgrade introduces a set of protocol changes for stateless validation. There are therefore two relevant protocol version voting dates. This doesn't require any action on the part of node operators, as your node will follow the current protocol version's rules as long as you upgrade. Validator nodes will vote first for protocol version 68, and then for version 69 after 68 is adopted.

Voting for upgrading to protocol version 68 will start on Tuesday 2024-07-09 17:00:00 UTC and voting for upgrading to protocol version 69 will start on Wednesday 2024-07-10 17:00:00 UTC

Congestion Control

Relevant NEP: near/NEPs#539
RPC API change: #11419

Congestion control introduces a feature that's meant to limit the transactions and receipts included in chunks when a shard is under load. Before this protocol change, when a particular shard is under heavy load, users can often experience frustrating experiences where transactions take a long time to be included because there is a very long queue of delayed receipts that take up chunk space. When this delayed receipt queue grows to an unreasonable size, it can be difficult for users to understand when things will settle down and their transactions will begin to be included again.

The congestion control feature adds fields to the chunk header that indicate the load on the corresponding shard in terms of delayed and outgoing receipt gas, and the protocol rules limit transactions and receipts destined for congested shards, mitigating the problem of delayed receipt queues growing unbounded to unreasonable levels.

Things to note

Transactions can be rejected with a “ShardCongested” error if the chain is congested heavily

Stateless Validation

This protocol upgrade is the main reason for designating this release as version 2.0.0. It will alter the roles of validators within the network and fundamentally change the process of validating state transitions. Currently when a node (validator or not) receives a new block and downloads all its chunks, it applies the transactions and receipts contained in those chunks by looking up and modifying values in the state of each shard. This means nodes need to keep the full state of all shards on disk. The stateless validation protocol upgrade will introduce a change that allows nodes to check the validity of chunks without needing to keep a local copy of the full state. There will be two roles after the upgrade: chunk producers and chunk validators. Chunk producers hold the state of the shard they are assigned to in memory and produce chunks for that shard. Chunk validators do not maintain any state locally and rotate on every single block to verify the state transition of a shard. A specific node may function both as a chunk producer and a chunk validator in the network. Whether a node is a chunk producer is based on stake: top 100 nodes by stake are chunk producers.

Changes to validator roles

Here is a high level summary of changes to validator roles

TL;DR;

  • All validators
    • Serve as chunk validators
    • Additional network usage is expected
    • Less disk usage is expected
  • Top 100 validators by stake:
    • In addition to chunk validator role, serve as chunk/block producers and block validators
    • Do not have to track all shards
    • State of tracked shard is loaded into memory
  • All other validators
    • Don’t need to track any shard

Overview of different validator roles

  • [NO CHANGES] Block producers (top 100 validators):
    • (Same as today) Produce blocks, (new) including waiting for chunk endorsements
    • (Same as today) Maintain chunk parts (i.e. participates in data availability based on Reed-Solomon erasure encoding)
    • (Same as today) Should have a high barrier of entry for security reasons, to make block double signing harder.
    • (New) No longer require tracking any shard
  • Chunk producers (top 100 validators):
    • (Same as today) Produce chunks
    • (Same as today) Must track the shard it produces the chunk for
    • (New) Produces and distributes state witnesses to chunk validators
  • [New] Chunk validators (all validators):
    • Validate state witnesses and sends chunk endorsements to block producers
    • Do not require tracking any shard
    • Must collectively have a majority of all the validator stake, to ensure the security of chunk validation.

Reward calculation

Reward calculation will remain the same and be based on online ratio.

Changes to contract requirements

  • New size limits were introduced to limit the size of state witness:
    • Transaction size must not exceed 1.5MB (used to be 4MB).
    • Receipt size must not exceed 4MB (possibly it may be reduced further down to 1.5MB).
    • A receipt is not allowed to read more than 4MB of state during a single function call.
    • Total size of transactions included in two consecutive chunks is limited to 4MB
    • Cross shard bandwidth is more limited than before
  • Increased cost of sending receipts
    • Sending a receipt to a shard (cross contract call) will now cost 50 TGas / MiB. Previously the cost varied between 2-18 TGas / MiB

Config changes

This section describes the configuration change a node operator needs to apply to their node before and after the stateless validation launch.
Let’s say that the current mainnet protocol version is X (before stateless validation).

When Top 100 validators, adopt the stateless validation release, they need to:

  • Change their nodes to meet hardware requirements (can be found below)
  • Mainnet: Set store.load_mem_tries_for_tracked_shards in config.json to true. Testnet: Set store.load_mem_tries_for_tracked_shards in config.json to true only after protocol version 68 is adopted by the network. otherwise nodes will crash attempting to load tries to memory
  • Set tracked_shards in config.json to [0] (This config will be updated after the protocol upgrade, but it should NOT be changed at this point.)
  • Recommendation: apply network optimization below

When Non-validator node operators, adopt the stateless validation release, they need to:

  • Change their nodes to meet hardware requirements (can be found below)
  • Set store.load_mem_tries_for_tracked_shards in config.json to false

The stateless validation release has two protocol upgrades, congestion control (protocol version 68), and stateless validation (protocol version 69). After these two protocol upgrades are done,

Top 100 validators must:

  • Downgrade their node to meet hardware requirements (can be found below)
  • Set tracked_shards in config.json to []
  • Recommendation: apply network optimization below

This means that validators only need machines with a lot of memory during the upgrade process.

All other validators must:

  • Set store.load_mem_tries_for_tracked_shards in config.json to true
  • Set tracked_shards in config.json to []
  • Recommendation: apply network optimization below

Non-validator node operators do NOT have any further action needed.

Network optimization

sudo sysctl -w net.core.rmem_max=8388608 && sudo sysctl -w net.core.wmem_max=8388608 && sudo sysctl -w net.ipv4.tcp_rmem='4096 87380 8388608' && sudo sysctl -w net.ipv4.tcp_wmem='4096 16384 8388608' && sudo sysctl -w net.ipv4.tcp_slow_start_after_idle=0

Validator key hot swap

There is an option to change a node or its configuration with near-zero downtime.
Let’s assume we created a new node with desired configuration and we want to move the validator key from the current validator to the new node.

If this is before stateless validation launch, then the new node is configured with:

  • set tracked_shards in config.json to [0]

and we have to wait until the new node finishes state sync for all shards.

If this is after stateless validation launch, the new node is configured with:

  • set tracked_shards in config.json to []
    then we should also:
  • set tracked_shadow_validator in config.json to “<validator_id>”
    where <validator_id> is the validator account ID of the current validator, and wait until the new node finishes state sync for the shard assigned to the current validator.

Steps to move validator key from current validator to the new node:

  1. Copy .near/validator_key.json to the new node.
  2. Stop the current validator.
  3. Send a SIGHUP signal to the new node.

The new node will pick up the validator key without restart and it will start validating.

Hardware requirements

Validators:

  • RAM:
    • During transition time:
      • MainNet: 80+GB
      • TestNet: 40+GB
    • After transition:
      • MainNet: 32+GB
      • TestNet: 32+GB
  • Network bandwidth:
    • Recommended burst capacity is 1 Gbps up/down.
    • Sustained usage rate will be below 100 Mbps up/down.
  • Backup nodes
    • Hot-switch to validator-key
    • Track all shards (to be configured in config.json)
    • Before and after transition: Mainnet: 80+GB, Testnet: 40+ GB

RPC nodes

  • RAM:
    • If in-mem trie is enabled:
      • MainNet: 80+GB
      • TestNet: 40+GB
    • If in-mem trie is NOT enabled:
      • MainNet and TestNet 24+GB
  • No changes to disk size and network bandwidth requirements

Archival nodes

  • RAM:
    • If in-mem trie is enabled:
      • MainNet: 80+GB
      • TestNet: 40+GB
    • If in-mem trie is NOT enabled:
      • MainNet and TestNet 24+GB
  • No changes to disk size and network bandwidth requirements
Check out latest releases or
releases around near/nearcore 2.0.0-rc.1

Don't miss a new nearcore release

NewReleases is sending notifications on new releases.

Get notifications