Mainnet 17

Major Changes

Updated to Cadence version v0.23.0, which contains a breaking change for the Public Key API

There has also been a lot of improvements made to the mTrie/merkle tree storage methods, specially made around the Write Ahead Log, and Checkpointing logic. The speed improvements here are mostly noticeable for node operations side and will not be DAPP facing.

There will be an internal concept change, which will better define the concept of "Weight" in the protocol. This changed required some renaming of Stake → Weight (Node level breaking change). This means that the read the new protocol state, it will require a matching node version that has the same conceptual understanding of Stake vs Weight.

Lastly, there is a change that will make to add the code necessary for Variable Transaction Fees, but this feature will not be enabled until a community vote has been held.

All Changes

💥 Breaking Changes

• Optimize MTrie Checkpoint (regCount & regSize): -9GB alloc/op, -110 milllion allocs/op, -4GB file size (#2126) @fxamacker
• Reduce checkpoint file size by using fewer bytes to encode length of encoded payload value (#2165) @fxamacker
• Update checkpoint file format version from v4 to v5 (#2174) @fxamacker
• Optimize MTrie checkpoint: 47x speedup (11.7 hours -> 15 mins), -431 GB alloc/op, -7.6 billion allocs/op, -6.9 GB file size (#1944) @fxamacker
• Rename Stake to Weight (#2090) @jordanschalm
• Cadence upgrade (#1975) @janezpodhostnik

Cadence & FVM

• [FVM] Variable Computation Metering (#1631) @janezpodhostnik
• [FVM] Change fee deduction logic (#2097) @janezpodhostnik
• [Execution] enable cadence runtime tracing given node level flag (#2108) @ramtinms
• [FVM] refactor computation and memory metering (#2129) @ramtinms
• [Migration] Migration for Ordered Map storage change (#1839) @dsainati1
• [FVM] relax tag conditions for ECDSA runtime signature verification (#1992) @ramtinms
• [FVM] Add bls crypto functions to transaction and script env (#1605) @dsainati1
• [FVM] Execution effort weights option for bootstrap (#2228) @janezpodhostnik
• [FVM] Move execution intensities debug logging (#2229) @janezpodhostnik
• [FVM] Fix metering limits (#2217) @janezpodhostnik
• [Cadence] update to Cadence v0.23.3 (#2247) @turbolent
• [FVM] Temporary fix for addressing storage used of a non-existing account (#2253) @janezpodhostnik

BFT Testing

• Implements attack network (#2144) @yhassanzadeh13
• Adds attacker implementation (#2121) @yhassanzadeh13
• Adds orchestrator interface and attack network (#2109) @yhassanzadeh13
• Implements Corruptible Conduits (#2033) @yhassanzadeh13

Access API

• Store and retrieve transaction results by EN RPC (#2166) @m4ksio
• Create transaction result by index api (#2159) @koko1123
• Add bootstrap command to generate observer networking key (#2087) @peterargue
• Create light block response as default for AN block lookup API (#2148) @koko1123
• Add Converters between RPC and flow-go Block with nested types (#2056) @koko1123
• Create unit test for gRPC ExecutionResult response structure (#2006) @koko1123
• Support running observer standalone using standard AN image (#2037) @peterargue
• Log unique cadence scripts in AN execution engine (#1934) @koko1123
• Add ExecutionDataId field in handler (#2010) @koko1123
• [HTTP API] Transaction success status (#2106) @sideninja

🛠 Node Improvements

• [Execution] Log execution data ID (#2196)(#2055)(#2039) @smnzhu

• [Execution] Add logging for CPU, RAM, and checkpoint (#2119)(#2187) @fxamacker

• [Execution] Reduce default checkpoint distance flag from 40 to 20 to speedup EN startup (#2223) @fxamacker

• [Execution] Remove transaction results from execution data (#2098) @smnzhu

• [Consensus] VoteAggregator pruning (#2040)(#2110) @durkmurder

• [Consensus] Epoch test suite updates (#2041) @kc1116

• [Consensus] Refactor epochs related contract deployments, use blueprints pattern (#2057) @kc1116

• [Consensus] ecretsdb key soft enforcement (#2042) @kc1116

• [Collection] Add message queue to transaction ingest engine (#2035) @jordanschalm

• [All Nodes] Root snapshot QC validation (#2038) @durkmurder

• [All Nodes] Performance: improve the binary search speed (#1493) @jwinkler2083233

• [All Nodes] Implement --profiler-mem-profile-rate= (#2059) @simonhf

• [Execution] Merkle trie updates (#1909) @tarakby

• [Consensus] DKG Messaging Improvements (#1950) @jordanschalm

• [All Nodes] Performance: replace calls to Lookup with binsearch (#1513) @jwinkler2083233

• [All Nodes] Add func that will check for snapshot validity (#1940) @kc1116

• FVM error detection (#2028) @sideninja

🐞 Bug Fixes

• Fix for missing cache (#2043) @jwinkler2083233
• Fix bootstrap cluster qc generation (#1976) @jordanschalm
• Fix potential overflow in ledger metrics LatestTrieRegCountDiff() and LatestTrieMaxDepthDiff() (#2096) @fxamacker
• Fix race in epoch switchover handling (#1980) @jordanschalm
• [HTTP API] Authorizers bugfix (#2152) @sideninja

CI/CD

• Flaky Test Monitor - BigQuery update and full refactoring of unit tests (#2102) @gomisha
• Add observer as a separate docker image (#2113) @m4ksio
• Avoid setup/teardown cycle for skipped epoch integration tests; re-enable base Epoch Transition test (#1881) @jordanschalm
• Fix non-x86 linter issue (#1938) @tarakby
• changing go mod to point to the latest OpenAPI spec (#1962) @vishalchangrani

Misc

• Replace use of deprecated ioutil funcs in ledger (#2084) @fxamacker
• [Observability] Adds metrics for HeroCache (#2093)(#2218) @yhassanzadeh13 @smnzhu
• [Admin] GetIdentity command (#2191) @smnzhu
• Update sync protocol flip (#2123) @smnzhu
• Target westmere processor when compiling relic for better compatibility and performance (#1993) @yagop
• [Admin] Remove double instantiation of admin server for access node (#2141) @smnzhu
• [Util CLI] Removing transit push command; adding transit upload-transit-keys com… (#2122) @vishalchangrani
• [Admin] Add admin command to enable/disable profiler (#2076) @zhangchiqing
• [Admin] Define ToggleUploaderCommand (#2114) @smnzhu
• [Observability] Configurable metrics (#2095) @m4ksio
• Update flow-go/crypto dependency (#2107) @janezpodhostnik
• Protocol randomness use cleanup (#2001) @tarakby
• Include core contract changes in cadence-issue-1127 (#2103) @janezpodhostnik
• [Network] Add read and write deadlines for direct messages (#2079) @smnzhu
• [Util CLI] Partner node info generation tool (#1988) @kc1116
• [Network] add new metric to track number of Unicast calls in progress (#2077) @smnzhu
• [Network] add metrics and log to track blocked queue workers (#2069) @smnzhu
• [Admin] implement read execution data admin command (#1998) @smnzhu
• [Util CLI] fungible token tracker (#2012) @ramtinms
• [Logging] Fix TransactionContractFunctionInvoker error log (#2031) @janezpodhostnik
• [Logging] Make ContractHandler Commit produce deterministic errors (#2003) @janezpodhostnik
• [Logging] log vote's signer and block when vote is invalid (#1982) @zhangchiqing
• [Networking] Implements Pluggable Conduit Factory (#1986) @yhassanzadeh13
• [Util CLI]Fix account reporter (#1972) @zhangchiqing
• [Testing] Allow bench-net permissionless contract deployment (#1970) @janezpodhostnik
• Split CI integration job into categories (#2242) @smnzhu
• Update request size limit for sync engine (#2245) @smnzhu
• Fix incorrectly skipped test (#2241) @smnzhu
• Fix output of fungible token tracker test (#2237) @smnzhu
• Test monitor GitHub actions (#2151) @smnzhu
• [BFT Testing] Implements integration tests for corruptible conduit factory framework (#2203) @yhassanzadeh13