Mainnet 19

Major items going out this release.

💥 Breaking Changes

• [Execution] avoiding double encoding of event key before insertion into the event merkle trie (#1994) @ramtinms
• [FVM] merge constant size account registers in account status (#2799) @ramtinms
• [FVM] remove the use of the legacy controller by the FVM registers (#2585) @ramtinms
• Crypto: BLS ciphersuite updates (#2576) @tarakby (announcement)

Execution node checkpointing improvements

• [Execution Node] Reuse ledger state in checkpoints for -152GB RAM and -24 minutes (#2792) @fxamacker
• [Ledger] Replace LRU cache with a FIFO queue (circular buffer) (#2893) @ramtinms
• Refactor root checkpoint extraction program (#2206) @fxamacker
• Fix copylocks in TestOnBlockProposal_RemoteOrigin (#2693) @fxamacker

Execution node

• [EN Performance] Reduce memory used for ledger.Payload by 32+ GB, eliminate 1+ billion allocs/op, speedup various ops (#2930) @fxamacker
• [Execution] Data storage updates (#2292 ) @smnzhu
• [Execution] re-organizing uploader code in Execution engine (#2854) @Tonix517
• [Execution] a fix to atree reporter (#2839) @ramtinms
• [Execution] adding more debug logs (#2710) @smnzhu
• [Execution] check if state commitment exists before executing scripts (#2763) @ramtinms
• [Execution] Fixed EN debug docker image build failure (#2708) @Tonix517
• [Exec] Remove global lock from script execution (#2742) @SaveTheRbtz
• [Exec] Add manager benchmark (#2668) @SaveTheRbtz
• [Exec] improve result upload/eds tracing (#2655) @SaveTheRbtz
• [Exec] Remove Stop The World pauses from transaction/script execution (#2739) @SaveTheRbtz
• minor EN stat cleanup (#2643) @SaveTheRbtz
• [Exec] add tracing to collection view merge (#2656) @SaveTheRbtz

Cadence & FVM

• Run fvm benchmark transactions under a regular account (#2894) @pattyshack
• Move weighted Meter into meter package (#2895) @pattyshack
• [FVM] Deduct fees and limit storage in synthetic benchmarks (#2881) @SaveTheRbtz
• [FVM] Deduct fees on all networks by default (#2880) @SaveTheRbtz
• [FVM] cleanup benchstat installation (#2884) @SaveTheRbtz
• [FVM] cleanup and regenerate mocks (#2858) @SaveTheRbtz
• [FVM] Update cadence to support Executor pattern (#2853) @SaveTheRbtz
• dedup code in script/transaction environment (part 2 of many) (#2845) @pattyshack
• Split env context, unsafe rand gen, and program logger into logical units (#2849) @pattyshack
• [FVM] All service accounts should be bootstrapped with a key (#2705) @janezpodhostnik
• [FVM] removing legacy controller from storage (#2713) @ramtinms
• [FVM] Fix error handling on contract function invocations (#2531) @janezpodhostnik
• [FVM] Don't read context from state on system chunk - port (#2690) @janezpodhostnik
• [FVM] Benchstat CI improvements (#2748) @janezpodhostnik
• [FVM] Add missing tests for merging FVM state weights (#2530) @janezpodhostnik
• [FVM] contracts.go refactor (#2353) @janezpodhostnik
• [FVM] fix a bug with invalid signature error type (#2424) @tarakby
• [FVM] add extra flag for contract removal (#2584) @ramtinms
• [FVM][TEST] Splitting fvm_test.go (#2594) @Tonix517
• Fix Cadence code (#2653) @turbolent
• Switch to cadence v0.25.0 by @SupunS in #2913
• Update to Cadence 5a06fec (#2709) @turbolent

Epoch and DKG Updates

• [Fix] Update epoch test suite teardown (#2627) @kc1116
• fixed decoding error for signer indices at epoch switchover (#2830) @AlexHentschel
• Unskip flaky test - TestEpochJoinAndLeaveLN (#2802) @LLsq2
• fix flaky test: TestStaticEpochTransition (#2698) @gomisha
• [fix:TestEpochJoinAndLeaveAN] [fix:TestEpochJoinAndLeaveVN] Handle registering node without machine account (#2699) @jordanschalm

Networking

• [Networking] Fixes flakey DNS cache expiry test (#2670) @yhassanzadeh13
• Khalil/ 1725 authenticate libp2p peer before decoding message payload (#2667) @kc1116
• Khalil/2706 update middleware errors (#2726) @kc1116

Consensus / BFT

• [Consensus] Conflicting seals mitigation strategy (#2892) @durkmurder
• Update Consensus Loop (#2857) @danielholmes839
• [Sealing Segment] Handle root sealing segment with multiple blocks (#2786) @jordanschalm
• 1714 add rate limiting interceptor to the LN & EN gRPC API's (#2505) @kc1116
• 1704 add impersonation/masquerade slashing hooks (#2648) @kc1116

Sync engine

• [SyncEngine] not request pending ancestors (#2879) @zhangchiqing
• Add exponential backoff for execution data download timeouts (#2629) @peterargue

Crypto

• Update crypto scripts for Go 1.18 (#2911) @tarakby
• crypto: allocate on stack where possible (#2598) @SaveTheRbtz
• Update crypto note in README (#2781) @tarakby
• [Crypto] fix memory leak in unhappy path (#2604) @tarakby
• Crypto: BLS ciphersuite updates (#2576) @tarakby

Access and Observer node

• Observer Pull Request Candidate 0603z (#2554) @szegedim
• Observer integration tests (#2783) @danielholmes839
• AN - Lock Updates to Conn Pool (#2779) @lolpuddle
• [AN] - Trigger Cache Eviction on Overwrite (#2760) @lolpuddle
• Invalidate Pooling Logic in AN (#2691) @lolpuddle
• AN - Lock Updates to Conn Pool (#2779) @lolpuddle
• [Access] add signer ids to access API (#2762) @zhangchiqing
• Improve error reporting for access api event endpoints (#2822) @peterargue
• Add warning logs to request handler (#2671) @danielholmes839
• Add latest height metrics for execution receipts and collections (#2692) @danielholmes839
• Fix GetTransactionResultsByBlockID Panic (#2729) @
• Fix time.Sleep (#2721) @
• Add a separate channel name for push-blocks on the public network (#2641) @peterargue

Performance improvements

• Stop type casting meter before setting weights / limits (#2908) @pattyshack
• [FVM] Storage check optimisation (#2837) @janezpodhostnik
• [FVM] reducing register touches (#2502) @ramtinms

BFT Testing

• [BFT Testing] Fixes wintermute orchestrator race condition (#2780) @yhassanzadeh13
• [BFT Testing] Add BFT tests to regular CI workflows (#2725) @gomisha
• [BFT Testing] refactors gRPC interface (#2599) @yhassanzadeh13
• [BFT Testing] Wintermute attack integration test (#2463) @yhassanzadeh13

Profiler/Telemetry changes

• expose StartSpanFromRoot AccountFreezeEnabled methods to environment interface (#2900) @pattyshack
• [Profile] disable threadcreate profiler (#2898) @SaveTheRbtz
• [Profiler] move out of ./utils/debug to ./module/profiler (#2897) @SaveTheRbtz
• [Profiler] Force GC before running heap profile (#2860) @SaveTheRbtz
• [All] OpenTracing to OpenTelemetry migration (#2823) @SaveTheRbtz
• [Profiler] Automatic pprof profile uploads to Google Cloud Profiler. (#2903) @SaveTheRbtz

🛠 General Node Improvements

• [Collection Node] Add dry-run option for collection rate limiting (#2615 ) @jordanschalm
• Port of complete removal of extralogs (#2682) @m4ksio
• Backport setExecutionParameters from TransactionEnv to ScriptEnv (#2835) @pattyshack
• Adding new load test type for constant sized transactions (#2816) @Tonix517
• Add ComputationMeter interface to facilitate env code dedup-ing (#2831) @pattyshack
• Fix Unit race condition on stopping. (#2775) @pattyshack
• Remove GO111MODULE=on everywhere (#2821) @SaveTheRbtz
• chore(comment): fix IsSampled description (#2809) @SaveTheRbtz
• chore(tests): remove unused mock (#2824) @SaveTheRbtz
• chore(tests): add trace benchmark (#2806) @SaveTheRbtz
• Add staging net (#2793) @Kay-Zee
• dedup code in script/transaction environment (part 1 of many) (#2790) @pattyshack
• chore(trace): rename file (#2810) @SaveTheRbtz
• Invoke contracts directly instead of via currying (#2804) @pattyshack
• chore(localnet): fix observer's trace endpoint (#2805) @SaveTheRbtz
• chore(loader): fix Stop/Follow race (#2766) @SaveTheRbtz
• change unittest fixture to use *flow.Header instead of flow.Header (#2744) @pattyshack
• Add Andrew Meyer to performance stream (#2769) @AndrewM-SDET
• [localnet] Adding checking to EXECUTION configuration (#2788) @Tonix517
• s/Rem/Remove/g to improve code readability (#2784) @pattyshack
• chore(loader): wait for account creation results (#2778) @SaveTheRbtz
• Extend consensus follower config to support compliance settings (#2697) @jordanschalm
• Update dependency modules version and update BLS DSTs (#2603) @tarakby
• chore(loader): faster worker Stop() (#2765) @SaveTheRbtz
• chore(loader): switch from Client to Access API (#2764) @SaveTheRbtz
• loader: remove TxTracker and TxStatsTracker (#2632) @SaveTheRbtz
• Don't leak signerLock outside of Account encapsulation (#2644) @pattyshack
• remove disk size metrics (#2746) @zhangchiqing
• tweak Programs locking + minor refactoring (#2749) @pattyshack
• Fixed docker-build-loader image build failure (#2757) @Tonix517
• chore(localnet): faster account creation (#2683) @SaveTheRbtz
• Remove header caching (#2704) @SaveTheRbtz
• optimize emergency-sealing check (#2673) @zhangchiqing
• Unskipping passing flaky test - TestComponentConsumerSuite/TestHappyPath/runs_and_notifies_using_pre-notifier (#2730) @LLsq2
• Moving emulator-based tests to the integration package (#2738) @ramtinms
• Fix reversed numbers for script metrics (#2595) @dsainati1
• Simplify blockCommitter/eventHasher (#2660) @pattyshack
• chore(execution): fix execution node histograms (#2651) @SaveTheRbtz
• Don't leak blockCommitter/eventHasher gorountines on error (#2661) @pattyshack
• Size args slice correctly (#2659) @pattyshack
• Override cbor DecMode config in execution data requester (#2674) @peterargue
• [AdminTool] Admin require approvals (#2518) @zhangchiqing
• Mark network.Engine as deprecated (#2600) @jordanschalm
• minor cleanup of sealing segment and fixing typos (#2605) @AlexHentschel
• [State Exporter] support pre-checkpointing reporter (#2352) @ramtinms
• Move script execution log to debug (#2663) @Kay-Zee
• chore(localnet): reduce copy-paste in bootstrap (#2650) @SaveTheRbtz
• log node id and block id when signing root block (#2626) @zhangchiqing
• chore(localnet): force uid within the container to be root (#2649) @SaveTheRbtz
• Add dht routing table size metric (#2639) @smnzhu
• fix get transactions edge case for root block (#2611) @smnzhu
• chore(loader): cleanup logging (#2628) @SaveTheRbtz
• 1706 refactor network codec as argument (#2491) @kc1116
• feat(loader): do not depend on flow-cli for feedback (#2488) @SaveTheRbtz
• Cleanup remote debugger code (#2577) @SaveTheRbtz
• Add more profile types to the profiler (#2579) @SaveTheRbtz
• Add larger buckets for memory estimate metrics (#2580) @dsainati1
• Log highest executed block height (#3070) @zhangchiqing