Mainnet 15

Two major items going out this release.

The first will be the introduction of the new Cadence Storage Layer. This new storage layer will make use of the atree, which will greatly improve the performance of storage interactions with extremly large storage paths. As part of this change, there is a breaking change in relation to the owner field. More on that in the forums

The second being automated Epoch transitions. Epochs will now auto transition once a week with an un-changing node operator list. The network will run the DKG and shuffle clusters each epoch, but node operator changes will not be enabled at epoch boundaries yet. Note that this does not yet include the reward payout, which will still be done manually until we are satisfied with the reliability of the auto epoch transitions.

Breaking Change

• bugfix for Cadence's resource owner field (link)(https://forum.onflow.org/t/breaking-change-bugfix-for-cadence-resource-owner-field/2576)

Cadence & FVM

• [Cadence] Bump Cadence to v0.20.2 (#1552)(#1676)(#1714) @SupunS @turbolent @Kay-Zee
• [Mtrie] Improve ledger write speed for newly allocated slabs (#1501) @ramtinms
• [FVM] Update ledger interaction limit for bootstrapping (#1648) @ramtinms
• [FVM] fix error messages (#1665) @tarakby
• [FVM] Recover ledger interaction error (#1625) @janezpodhostnik
• [Mtrie] Fix for Mtrie bug (#1633) @AlexHentschel
• [FVM] Clean up FVM (#1613) @turbolent
• [Mtrie] Trie pruning (#1578) @ramtinms
• [FVM] Interaction limit related issues(#1504)(#1592) @janezpodhostnik @ramtinms
• [Mtrie] On update trie pruning (complete ledger) (#1507) @ramtinms
• [FVM] Change meta script invocation to direct contract function calls (#1318) @janezpodhostnik
• [FVM] Fix fees not being deducted on gas limit reached (#1367) @janezpodhostnik

Epoch and DKG Updates

• [DKG] Do not crash for DKG failures (#1693) @tarakby
• [Epoch] Update AN builders and scaffold to use spork id (#1696) @kc1116
• [Epoch] Epoch counter reporter (#1691) @janezpodhostnik
• [DKG] Remove unnecessary fields from DKG Private Info (#1673) @tarakby
• [Epoch] Refactor fallback logic for qc voter & dkg broker (#1556) @kc1116
• [Epoch/DKG] Add jitter to all QC/DKG requests (#1637) @jordanschalm
• [Epoch] Update reset-tx-args command (#1615) @jordanschalm
• [Epoch] Epoch Emergency Fallback Improvements (#1569) @jordanschalm
• [Epoch] SporkID and ProtocolVersion (#806)(#810) @jordanschalm
• [Epoch] Extend root snapshot (#1533) @kc1116
• [Epoch] Use SporkID for network channel namespacing (#1474) @jordanschalm
• [Epoch] Use latest finalized (not sealed) reference block (#1607) @jordanschalm
• [DKG] Fixes a bug where DKG validation would continue after encountering a fatal error (#1583) @jordanschalm
• [Epoch] Add func to set approved nodes list (#1478) @kc1116
• [Epoch] Remove redundant key generation code (#1469) @kc1116

Networking

• [Networking] Update libp2p-tls to fix issue with secured gRPC certificate (#1694) @peterargue
• [Networking] Stream negotiation and fallback (#1643) @yhassanzadeh13
• [Networking] Update connection manager implementation (#1516) @smnzhu
• [Networking] Unstaked node networking updates (#1506) @smnzhu
• [Networking] Tweak-able networking received cache size (#1529) @yhassanzadeh13
• [Networking] Changing multicast to use the filtered set of node ids (#1528) @vishalchangrani
• [Networking] Logging dial addresses (#1492) @yhassanzadeh13
• [Networking] Add bitswap API (#1356) @smnzhu
• [Networking] Middleware component (#1355) @smnzhu
• [Networking] Libp2p stream compression (#1349) @yhassanzadeh13

General Node and Crypto Improvements

• [Access] Adding a block height threshold to re-request missing collections (#1711) @vishalchangrani
• [Access] Adding middlewares to the REST API implementation (#1619) @vishalchangrani
• [Execution] Using valid descendents when loading unexecuted blocks (#1588) @zhangchiqing
• [Access] REST API implementation - GetBlockByID (#1524) @vishalchangrani
• [Verification] Removes less and swap methods of chunk list (#1608) @yhassanzadeh13
• [BFT] Error handling for engines (#1568) @durkmurder
• [State Sync] state serialization module (#1483) @smnzhu
• [Consensus] Continue recover block when hitting invalid block (#1564) @zhangchiqing
• [Consensus] sealing visualization (#1457) @smnzhu
• [Crypto] light SHA3 API (#1359) @tarakby
• [Crypto] Improve Invalid input errors (#1553) @tarakby
• [Consensus] Remove legacy code in Ingestion engine (#1532) @AlexHentschel
• [Consensus] Ingestion engine inbound messages queue (#1517) @durkmurder
• [Consensus] Fix potential mem-leak from hotstuff (#1514) @zhangchiqing

🛠 Misc. Improvements

• [Misc] Do not use pointer receiver for MarshalJSON (#1652) @SaveTheRbtz
• [Misc] Update minimum go version to 1.16, and default to 1.17 (#1692) @peterargue
• [Misc] Fix code ownership for access (#1653) @SaveTheRbtz
• [Metrics] Fix randomness in execution's example (#1651) @SaveTheRbtz
• [Misc] Reenable Reporters (#1410) @janezpodhostnik
• [Admin] Add commands for reading entities (#1549) @smnzhu
• [Admin] Update live debugger to include block header data (#1587) @ramtinms
• [Misc] Adds an additional error check (#1661) @Kay-Zee
• [Documentation] Updating broken links to node docs (#1558) @j1010001
• [Verification] makes verification node fork-aware (#1538) @yhassanzadeh13
• [Documentation] Adding section on playing with localnet (#1531) @j1010001
• [Metrics] Disable cache metrics (#1518) @zhangchiqing
• [Misc] removing rlp-encoded private key string to use flow.AccountPrivateKey (#1525) @j1010001
• [Documentation] Fixing broken links in access node docs (#1481) @j1010001
• [Misc] Clean up secondary result index (#1515) @Kay-Zee
• [Misc] Adding log tracer module for debugging purposes (#1496) @ramtinms
• [Admin] Adding rpc endpoint to get registers from exec nodes (#1460) @ramtinms
• [Admin] Adding transaction and script debugger using remote registers (#1468) @ramtinms
• [Admin] add log level customization command (#1362) @smnzhu
• [Misc] Simplify address conversion (#1422) @turbolent
• [Misc] Transaction model status bugfix (#1646) @sideninja

CI & Tests

• [CI] Fix localnet docker build caching (#1712) @simonhf
• [Tests] Unskipping the TestGhostNodeExample tests (#1670) @vishalchangrani
• [CI] Add github action to make a benchstat comparison of a PR with its base (#1579) @janezpodhostnik
• [CI] Flaky test monitor (#1495)(#1500)(#1526) @gomisha @smnzhu
• [Networking] fixes flakey stream closing test (#1600) @yhassanzadeh13
• [Tests] Fix flaky TestUnstakedAccessSuite/TestReceiveBlocks tests (#1551) @peterargue
• [Tests] Add test case for storage used (#1473) @SupunS
• [CI] fixes new line bug on verification integration tests (#1498) @yhassanzadeh13
• [CI] Run CI on PRs targetting version branches (#1443) @jordanschalm
• [Tests] Add more tests for value converter (Storage Migration V6) (#1401) @SupunS