Protocol Changes

• Voting for the protocol upgrade is scheduled to start at 2022-09-28 15:00:00 UTC.
• Stabilized protocol_feature_chunk_only_producers. Validators will now be assigned to blocks and chunks separately.
• The validator uptime kickout threshold has been reduced to 80%
• Total stake of kicked out validators will not exceed 30% of total stake

Features:

• The logic around forwarding chunks to validators is improved
• Approvals and partial encoded chunks are now sent multiple times, which should reduce stalls due to lost approvals when the network is under high load
• We now keep a list of "TIER1" accounts (validators) for whom latency/reliability of messages routed through the network is critical
• /debug HTTP page has been improved
• Messages aren't routed through peers that are too far behind
• network.external_address field in config.json file is deprecated.
• More useful metrics around message handling have been added
• Data from the DB is now prefetched while applying chunks

Fixes:

• Log lines printed every 10 seconds are now less expensive to compute
• message broadcasts have been improved/optimized
• Contention in the peer actor/networking code has been reduced.
• A bug that made chain info unavailable to the client at startup has been fixed.
• Improved shard cache for Trie nodes to put more nodes in memory. Requires additional RAM memory.
• The logic involved in processing chunk parts that have been forwarded to a validator has been improved

CODE_COLOR: CODE_YELLOW_MAINNET
RELEASE_VERSION: 1.29.0
PROTOCOL_UPGRADE: TRUE
DATABASE_UPGRADE: FALSE
SECURITY_UPGRADE: FALSE

More Information

Chunk-only producers

The 1.29 release introduces Chunk-only producers, an important change in the protocol that improves NEAR’s sharding capabilities and contributes to further decentralization of the protocol.

What chunk-only producers mean for validators? The number of mainnet validators will increase: currently mainnet has 100 validators, and after this protocol change, we’ll have 100 “block producers” and 200 “chunk-only producers”. In total, there will be up to 300 active participants in the blockchain.

Role assignment is based on stake: the protocol will select the top 100 nodes (based on the stake) and assign them as “block producers”. The following 200 nodes (again, based on stake) are assigned as “chunk-only producers”.

Each chunk in a block will be produced by a different producer - so there will be 5 independent producers participating in every block production (4 chunk producers and 1 block producer). This might increase the number of cases where a block is missing a few chunks. Currently (in 1.28) the same entity produces the block and all the chunks - so usually either the whole block is missing, or all chunks are present.

Configuration Changes

[MANDATORY] tracked_shards

Make sure that you have "tracked_shards" set:

"tracked_shards": [0],

In your config.json. If you don’t - the new binary will probably panic on start ;-)

The reason why this is needed is the following: with chunk-only producers, the algorithm itself no longer requires everyone to produce chunks from every shard. You’ll notice that most of the validators will be assigned to a single shard.

But, for security reasons, we have to make sure that all the validators are tracking all the shards. This approach will be maintained until we complete Phase 2 of sharding (that includes Challenges, roll-back support, etc).

[OPTIONAL / Will be mandatory in the future] Public IP addresses

As a part of strengthening the network, we’ll slowly be moving towards requiring public IP addresses for the validators (or running a proxy node with public IPs).

In the simplest setup, this will require you to set:
public_addrs = [“ed25519:YOUR_NODE_KEY_JSON_PUBLIC_KEY:YOUR_IP_ADDRESS:PORT”]
Within the network tuple in config.json.

Example:
["ed25519:86EtEy7epneKyrcJwSWP7zsisTkfDRH5CFVszt4qiQYw@31.192.22.209:24567"]

You can see more information about this config option here:

In this release, public IP addresses are still optional. Don’t hesitate to contact us if you have any questions about this.

[OPTIONAL] Enable Debug RPC

In this release we’ve added many debug features that will allow you to quickly see what’s happening with your node). You can find instructions about them here:
https://nearinc.atlassian.net/wiki/spaces/DOCS/pages/139296992/Debug+RPC+and+debug+pages

To enable the debug features, you have to set:
"enable_debug_rpc": true

in your config.json file.

Note: This is also a great place where you can contribute to protocol development. If you see some more things that should be added to the debug pages, don’t hesitate to create a github issue and/or implement it yourself.

[OPTIONAL] Sharing metrics with Pagoda

This is very much optional, but we’d like to ask you to allow us to scrape the metrics from your validator node. Doing so would allow us to detect and debug mainnet issues a lot faster - and we’ll be able to share back some aggregated metrics back with the community.

Metric sharing can be done by whitelisting 2 IP addresses that we’ll use for our prometheus scraper. More information in:
https://nearinc.atlassian.net/wiki/spaces/DOCS/pages/149979169/Exposing+validators+metrics+to+Pagoda

If you would like to enable sharing, please follow the steps in the link above, and fill-in this form: https://forms.gle/rQwwe9q4PKuUmGAx9

[FYI] Changing log level at runtime

Did you know that you can change the neard log level at runtime? This allows you to enable more detailed logging, whenever something bad/suspicious happens with your node, without having to do a restart.

More info: https://nearinc.atlassian.net/wiki/spaces/DOCS/pages/149946369/Change+neard+log+level+at+runtime