sigp/lighthouse v4.4.1

Previous Leon

on GitHub

Summary

This is a low-priority release. It contains several minor bug fixes and performance improvements which we recommend rolling out on mainnet nodes at your convenience. This release also includes support for the Holešky testnet, with genesis on 15 September 2023.

Note that there is no v4.4.0 release due to a bug found late in the release cycle. The bug was fixed in PR #4687, which is included in this release.

The highlights of v4.4.1 are:

• [NEW] Introduction of the lighthouse validator-manager command for managing validators on a running validator client. Documentation can be found in a new chapter of the Lighthouse book: https://lighthouse-book.sigmaprime.io/validator-manager.html.
• Improved scheduling for HTTP requests (#4462). Lighthouse will now balance the load of HTTP requests against other important duties, and avoid becoming overwhelmed. We hope that this improves stability for nodes supporting a large number of validators or processing frequent HTTP queries.
• Disk space savings for new nodes through state pruning (#4610, #4663).
• Bugfix for backfill sync erroring upon completion (#4465).
• Support for the Holešky testnet via --network holesky (#4653).
• Support for the Chiado Gnosis testnet via --network chiado (#4530).

🐞 Known Issues 🐞

We have identified a minor bug in the code related to state pruning that can cause an error message to be logged for a limited time after upgrading:

ERRO Missing chunk in forwards iterator

It can be safely ignored, as it does not impact the node's ability to follow the chain or perform validator duties. However, API queries for historic blocks & states may fail until the issue resolves itself. This only affects nodes with an existing database. Nodes with lower --slots-per-restore-point will heal faster, and we expect the maximum time to heal to be around 8192 slots (~27 hours). The issue is being tracked in #4697.

⚠️ Breaking Changes ⚠️

Breaking Change: State pruning for new nodes

Newly synced nodes will no longer store historic states by default. This applies both to checkpoint sync and genesis sync. Node operators may opt-in to historic state storage by reconstructing states with the --reconstruct-historic-states flag.

Existing nodes will continue storing historic states newer than the checkpoint from which they synced. We are working on a command to prune historic states from existing nodes, which will likely be available in the next release.

The pruned status of a node can be determined using the /lighthouse/database/info API, which will show the value of state_upper_limit as 18446744073709551615. For more details see #4610, #4663.

Breaking Change: Inclusion delay in HTTP rewards API

Support for the phase0 hard fork was added to the attestation rewards API. This required the addition of a new field for the inclusion_delay, which has been standardised upstream in beacon-APIs: https://ethereum.github.io/beacon-APIs/#/Rewards/getAttestationsRewards.

See #4474, #4520 for details.

Breaking Change: HTTP status code for duplicate blocks

Lighthouse will now return an HTTP 202 ACCEPTED response when attempting to publish a block that is already known. This behaviour goes against the current beacon-APIs spec, but is likely better for the majority of API consumers. Previously, a 400 BAD REQUEST response could cause the caller (usually a validator client) to think that block publication had failed when it had not. We have begun a discussion to change the spec to allow the new error code.

The status code can also be set to a custom value using the flag --http-duplicate-block-status. See #4655 for details.

New HTTP APIs

The Lighthouse BN supports the following new HTTP APIs:

• SSZ variant of POST /eth/v{1,2}/beacon/blocks (#4479).
• SSZ variant of POST /eth/v{1,2}/beacon/blinded_blocks (#4504).
• POST /eth/v1/validator/liveness/{epoch} (#4343).
• GET /eth/v1/builder/states/{state_id}/expected_withdrawals (#4390).

Update Priority

This table provides priorities for which classes of users should update particular components.

See Update Priorities more information about this table.

The Beacon Node may be updated without also updating the Validator Client. However we recommend upgrading both components for consistency.

All Changes

• Release v4.4.1 (#4690)
• Remove double-locking deadlock from HTTP API (#4687)
• Release v4.4.0 (#4673)
• Pin foundry toolchain version to fix stuck CI jobs
• feat: add chiado (#4530)
• Fix bug in block root storage (#4663)
• Add Holesky (#4653)
• Increase web3signer_tests timeouts (#4662)
• Show --gui flag in help text (#4660)
• Do not log slot clock error prior to genesis (#4657)
• Send success code for duplicate blocks on HTTP (#4655)
• update dependencies (#4639)
• rust 1.72 lints (#4659)
• Use lockfile with cross and fix audit fail (#4656)
• Revive mplex (#4619)
• Implement expected withdrawals endpoint (#4390)
• Remove Antithesis docker build workflow (#4642)
• Fix the link of anvil in lighthouse book (#4641)
• Fix beacon-processor-max-workers (#4636)
• ForkChoice: remove an impossible case for parent_checkpoints (#4626)
• Remove checkpoint alignment requirements and enable historic state pruning (#4610)
• Fix Prefer Builder Flag (#4622)
• Update blst to 0.3.11 (#4624)
• add metrics::VALIDATOR_DUTIES_SYNC_HTTP_POST for post_validator_duties_sync (#4617)
• attester_duties: remove unnecessary case (#4614)
• Don't kill SSE stream if channel fills up (#4500)
• Fix disable backfill flag not working correctly (#4615)
• Improve HTTP API error messages + tweaks (#4595)
• ForkChoice: remove head_block_root field (#4590)
• opt maybe_update_best_child_and_descendant: remove an impossible case (#4583)
• Use the native concurrency property to cancel workflows (#4572)
• Fix bug of init_from_beacon_node (#4613)
• use account_manager::CMD instead of magic string (#4612)
• remove optional_eth2_network_config (#4611)
• minor optimize process_active_validator: avoid a call to state.get_validator (#4608)
• align editorconfig with rustfmt (#4600)
• Fix some typos (#4565)
• Improve transport connection errors (#4540)
• Use BeaconProcessor for API requests (#4462)
• Add validator-manager (#3502)
• Increase slashing protection test SQL timeout to 500ms (#4574)
• Support SSZ request body for POST /beacon/blinded_blocks endpoints (v1 & v2) (#4504)
• Update doppelganger note about sync committee contributions (#4425)
• Shift networking configuration (#4426)
• CommitteeCache.get_all_beacon_committees: set correct capacity to avoid realloc (#4557)
• CommitteeCache.initialized: fail early if possible (#4556)
• fix typo (#4555)
• upgrade to libp2p 0.52 (#4431)
• Deprecate exchangeTransitionConfiguration functionality (#4517)
• Update Lighthouse book FAQ (#4510)
• add ssz support in request body for /beacon/blocks endpoints (v1 & v2) (#4479)
• Development feature flag - Disable backfill (#4537)
• Add Eth Version Header (#4528)
• Fix incorrect ideal rewards calculation (#4520)
• Fix bug in lcli transition-blocks and improve pretty-ssz (#4513)
• Restore upstream arbitrary (#4372)
• Implement liveness BeaconAPI (#4343)
• Add self-hosted runners v2 (#4506)
• Phase 0 attestation rewards via Beacon API (#4474)
• Cleanup unreachable code in lcli::generate_bootnode_enr and some tests (#4485)
• Speed up CI by installing foundry with Github action (#4505)
• Rust 1.71 lints (#4503)
• CI fix: add retries to eth1 sim tests (#4501)
• Clean up local testnet files without prompting (#4498)
• Remove wget dependency (#4497)
• Update mev-rs and remove patches (#4496)
• Remove instances of required arguments with default values (#4489)
• Use unique arg names for eth1-sim (#4463)
• Fix HTTP state API bug and add --epochs-per-migration (#4236)
• bump proc-macro2 (#4464)
• Remove hidden re-exports to appease Rust 1.73 (#4495)
• Add state-root command and network support to lcli (#4492)
• Move the BeaconProcessor into a new crate (#4435)
• Bump default checkpoint sync timeout to 3 minutes (#4466)
• Correct checks for backfill completeness (#4465)

Binaries

See pre-built binaries documentation.

The binaries are signed with Sigma Prime's PGP key: 15E66D941F697E28F49381F426416DC3F30674B0

System	Architecture	Binary	PGP Signature
	x86_64	lighthouse-v4.4.1-x86_64-apple-darwin.tar.gz	PGP Signature
	x86_64	lighthouse-v4.4.1-x86_64-apple-darwin-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v4.4.1-x86_64-unknown-linux-gnu.tar.gz	PGP Signature
	x86_64	lighthouse-v4.4.1-x86_64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	aarch64	lighthouse-v4.4.1-aarch64-unknown-linux-gnu.tar.gz	PGP Signature
	aarch64	lighthouse-v4.4.1-aarch64-unknown-linux-gnu-portable.tar.gz	PGP Signature
	x86_64	lighthouse-v4.4.1-x86_64-windows.tar.gz	PGP Signature
	x86_64	lighthouse-v4.4.1-x86_64-windows-portable.tar.gz	PGP Signature
System	Option	-	Resource
	Docker	v4.4.1	sigp/lighthouse