Mainnet 16

Major Changes

No breaking changes for Cadence included in this update.

There are two major features included in this software upgrade:

• Consensus Voting V2: Protocol level optimization on how voting on blocks is performed, allowing more parallelism in vote processing
• Epoch Dynamic Node Addition: Adds ability to more easily adjust Protocol member list during Epoch transitions

There has also been a lot of improvements made to the mTrie/merkle tree implementation including a bug fix which will effect how the Payload hash is generated. There are also improvements to caching, such as adding a new cache type which we're currently calling HeroCache.

Lastly, there is a change that will make this software version relatively incompatible with previous Flow node software versions, which is adding the execution CID, in preparation for the improve state sync engine. This change adds an extra field to the ExecutionResult data structure, which will produce different hashes for the ExecutionResult compared to previous software versions

All Changes

Cadence & FVM

• Check and Use Contract Audit Vouchers (#1826) @tehranifar
• Fix LedgerGetRegister to avoid caching empty value (#1875) @fxamacker
• Improve error handling in MTrie checkpoint file creation (#1892) @fxamacker
• Support legacy Keccak 256 hashing algorithm (#1868) @ramtinms
• Backport: Ledger interaction error (#1878) @zhangchiqing
• Optimize Forest.Proofs() and add ValueSizes() to MTrie, Forest, and Ledger (#1870) @fxamacker
• Preallocate slices in MTrie (#1838) @fxamacker
• Update to Cadence v0.20.3 (#1829) @turbolent
• Add logs for MTrie checkpoint file creation (#1894) @fxamacker
• Update MTrie with some minor fixes and small improvements (#1801) @fxamacker
• Relaxing ledger interaction limit for service and system transactions (#1753) @ramtinms
• Tweak FVM bench to be more stable (#1722) @janezpodhostnik

Epoch and DKG Updates

• Dynamic Node Startup (#1893) @kc1116
• Improve handling of post-spork beacon key initialization (#1917) @jordanschalm
• Epochs integration test validation: consensus nodes (#1856) @kc1116
• Include staking auction length in resetEpoch (#1804) @jordanschalm
• Update deployEpoch for new FlowEpoch function (#1854) @jordanschalm
• Skip starting dkg reactor engine if node not part of dkg committee (#1849) @kc1116
• Enforce weight/ejection requirements in cluster committee (#1842) @jordanschalm
• Epochs test collection node validation (#1819) @kc1116
• Set epoch final view metrics on transition (#1806) @jordanschalm
• Epochs Integration testing: Verification node (#1763) @kc1116
• DKG Comments / Logging Changes (#1769) @jordanschalm
• DKG State (#1738) @jordanschalm
• DKG Storage Pt III: Check beacon key consistency atomically with epoch phase transition (#1732) @jordanschalm
• DKG Storage Pt II: Update components using DKGState storage (#1721) @jordanschalm
• DKG Storage Pt. I: Update storage layer (#1720) @jordanschalm
• Epochs integration tests (#1728) @kc1116
• Epochs test for joining with dynamic container addition/removal (#1603) @kc1116

Networking

• Refactors DNS cache with HeroCache (#1888) @yhassanzadeh13
• Refactoring networking received cache to HeroCache (#1908) @yhassanzadeh13

General Node and Crypto Improvements

• Update state synchronization (#1930) @smnzhu
• Chacha20-based PRG (#1782) @tarakby
• Clean up BLST tests (#1754) @tarakby

Access Node

• adding status_code to transaction result reponse in the REST API (#1959 ) @vishalchangrani
• Report execution errors for cadence scripts as OK in Access Node RPC calls (#1905) @koko1123
• Add naive error logging for Access node downstream calls (#1883) @koko1123
• REST api implementation (#1666) @vishalchangrani
• REST: Stability improvements (#1809) @sideninja

Cosensus Hotstuff

• Consensus Voting V2 (#1916) @zhangchiqing
• Sealing Core: Allow unverifiable block references below root height (#1828) @jordanschalm
• Extend Sealing Segment Structure (#1755) @jordanschalm

Sporking process improvements

• Improve state extraction (#1859) @zhangchiqing
• Validate the saved root snapshot (#1832) @zhangchiqing

Execution

• Improve transaction execution log (#1869) @zhangchiqing
• Switch eventList hash to use a Merkle trie (#1834) @ramtinms
• Better printing for keys in errors (#1781) @m4ksio
• Properly escape user-input in error messages (#1863) @m4ksio
• Log epoch number on execution node startup (#1731) @janezpodhostnik
• Remove metering computation twice (#1724) @janezpodhostnik

State sync engine

• Add execution data cid (#1775) @smnzhu
• Execution data requester (#1582) @smnzhu
• Sync engine / core updates (#1697) @smnzhu
• Allow registering custom node shutdown handlers (#1907) @peterargue
• Make NodeBuilder return runnable Nodes (#1543) @peterargue

Build, CI & Tests

• Fix flow-go failing to cross compile (#1939) @janezpodhostnik
• Fix ledger-heavy load generator (#1933) @janezpodhostnik
• Emulator build without Relic + Enable crypto no-Relic tests (#1922) @tarakby
• Fix undefined: sha3State errors when building on arm64 (#1903) @peterargue
• Support arm64 when building flow docker containers and localnet (#1904) @peterargue
• Disable machine account balance checking on test networks (#1877) @jordanschalm
• Improve Topology Factory for integration tests (#1880) @yhassanzadeh13
• Fix crypto test flakiness in randomness test (#1879) @tarakby
• Replace deprecated go get with go install (#1860) @koko1123
• Fix loader backport (#1841) @zhangchiqing
• Fix localnet loader for transaction execution success (#1816) @simonhf
• Flaky Test Monitor - CI fix - updated level 1 script to use level 1 sub-directory (#1808) @gomisha
• Stream negotiation and fallback for integration tests (#1812) @yhassanzadeh13
• Add intensive resources flag to AN tests (#1817) @kc1116
• Flaky Test Monitor - Level 3 Summary (#1793) @gomisha
• Fix bug in transfer tokens transaction in integration load tests (#1789) @janezpodhostnik
• Flaky Test Monitor - Summary Level 2 (#1707) @gomisha
• Use debian 11 (bullseye) (#1768) @Kay-Zee
• Skip flaky test conditionally (#1535) @smnzhu

🛠 Misc Improvements

• Limit range queries for sync engine @m4ksio
• Remove sudo from crypto setup (#1945) @tarakby
• Remove the crypto replace and add a crypto setup script (#1742) @tarakby
• Remove unnecessary allocations (#1596) @jwinkler2083233
• Refactors Transactions mempool to HeroCache (#1902) @yhassanzadeh13
• Adding proof to the merkle trie (#1833) @ramtinms
• Optimizes All method of BackData (#1852) @yhassanzadeh13
• Payload Patricia Trie Security fixes (#1800) @AlexHentschel
• Implements GC-optimized cache (#1778) @yhassanzadeh13
• Improve efficiency of Views and Heights gadgets (#1823) @jordanschalm
• Make hex address parsing consistent with SDK (#1792) @jordanschalm
• Removing legacy Identities interface (#1777) @AlexHentschel
• Refactor Merkle Patricia Tree used for payload hashing (#1678) @AlexHentschel
• Update min LN machine account balance (#1774) @jordanschalm
• Adding validation when constructing a Local module (#1704) @zhangchiqing
• Add readme for live debugger (#1716) @ramtinms