onflow/flow-go v0.39.0

Protocol Upgrade - EFM Recovery

on GitHub

What's Changed

⬆️ Coordinated Upgrades for all Node Roles

This is a mechanism built into the Protocol State which enables view-coordinated upgrades to the Protocol State data model. This mechanism can be leveraged to implement coordinated behavioural changes ("protocol upgrades") for all node roles, including for backward-incompatible upgrades. Previously such upgrades were supported only for Execution Nodes and the Execution stack via the Height Coordinated Upgrade mechanism and VersionBeacon smart contract.

This release completes the upgrade feature and implements the first such coordinated upgrade, which will be used to enable EFM Recovery (see below). Further details are in the working group meeting notes.

Related PRs

• KVStore Model Upgrade & Chunk.ServiceEventCount Model Upgrade Logic by @jordanschalm in #6796
• Chunk Data Model supports per-chunk service event mapping by @jordanschalm in #6744
• DKG Result Submission Backward Compatibility by @jordanschalm in #6827
• [BFT] Database migration for flow.DKGEndState by @durkmurder in #6861
• CBOR Encoding is compatible with stricter network decode mode by @jordanschalm in #6879

🌟 Epoch Fallback Mode Recovery

Epoch Fallback Mode (EFM) is a operational mode triggered when an epoch transition fails for any reason. While in EFM, the current epoch is extended in perpetuity and epoch transitions are disabled. This has the effect of disabling changes to the node operator committee. Also, automatic reward payout is disabled and reverts to a manual governance transaction.

EFM Recovery provides a mechanism to recover from epoch failures using a governance transaction. Previously, recovery from EFM required a network upgrade (spork). With EFM Recovery, a governance transaction is used to resume epoch transitions. Further details are in the design document.

EFM Recovery is available in this release, but must be enabled via a coordinated upgrade (see above).

Related PRs

• Feature: EFM Recovery by @jordanschalm in #6885
• [EFM] Added a service event to update EpochExtensionViewCount by @durkmurder in #6272
• [EFM] Telemetry consumers for KV store state machines by @durkmurder in #6291
• [EFM] Add EjectIdentity service event by @durkmurder in #6296
• [EFM] minor polishing by @AlexHentschel in #6309
• Updated documentation of FinalizationSafetyThreshold by @AlexHentschel in #6384
• [EFM] Renamed EpochCommitSafetyThreshold -> FinalizationSafetyThreshold by @durkmurder in #6318
• [EFM] Documentation of mapping from NodeID -> DKG index by @AlexHentschel in #6428
• [EFM] Extended documentation where generalized protocol logic meets bootstrapping implementation by @AlexHentschel in #6452
• [EFM] Update protocol.DKG to use IndexMap by @durkmurder in #6338
• [EFM] Update DKG engine to submit a valid DKG index map by @durkmurder in #6490
• [EFM] Updated type conversion of EpochCommit by @durkmurder in #6494
• [EFM] Enable previously disabled tests by @durkmurder in #6496
• [EFM] Updated type conversion of EpochRecover by @durkmurder in #6506
• EFM Recovery: Fixes from Benchnet Testing by @jordanschalm in #6620
• EFM Integration Test Part 2 by @kc1116 in #6424
• [EFM] Refactored encoding and decoding of DKG data during bootstrapping by @durkmurder in #6745
• updating the term DKG key as it is outdated by @AlexHentschel in #6735
• Random beacon comments by @tarakby in #6590
• [EFM] Recoverable Random Beacon State Machine by @durkmurder in #6771
• Re-enable test TestProtocolVersionUpgrade by @jordanschalm in #6752
• [EFM] EpochCommit is backward compatible by @durkmurder in #6795
• [BFT] Node ejected before epoch recovery by @durkmurder in #6632
• [BFT] Epoch Recovery integration test by @durkmurder in #6823
• [EFM] Recoverable Random Beacon State Machine follow up updates by @durkmurder in #6815
• [EFM] Fix bugs from Benchnet testing by @jordanschalm in #6898
• Use SafetyData to avoid double-proposing instead of myLastProposedView by @jordanschalm in #6921
• [EFM] Extra flags for the recovery command line tool by @durkmurder in #6935
• [EFM] Extra consistency checks for DKG Index Map by @durkmurder in #6936
• Add metric for current kvstore version by @jordanschalm in #6953
• [EFM] Unit tests for excluding/including recovery epoch participants by @durkmurder in #6968

🪨 Storage Layer

• [Storage Refactor] Init pebble DB in scaffold. by @zhangchiqing in #6949

⛰️ Access API

• [Access] Disable registers db pruning by default by @peterargue in #7017
• Validation of error message on transaction replay is broken by @m-Peter in #7026
• Generate random string with maxLen symbols instead of UUID by @illia-malachyn in #7007
• [Access] Update model Block REST API Response by @UlyanaAndrukhiv in #6884
• Add feature flag for new websockets stream API by @illia-malachyn in #7024
• Return error if --state-stream-addr no configured. by @illia-malachyn in #7010
• Updates core contracts and transactions to use correct import placeholders by @joshuahannan in #7027
• [Access] Tuning API logging by @peterargue in #7048
• Update EVM ViewProvider to return block View at its end state by @m-Peter in #7050

❓ Miscellaneous

• Rename MonotonousCounter to MonotonicCounter by @peterargue in #6998
• Refactor Consensus Sealing Engine+Core: engine.Unit -> ComponentManager by @tim-barry in #6934
• Use ExecutionParameters contract in bootstrapping by @joshuahannan in #6986

Full Changelog: v0.38.3...v0.39.0