ObolNetwork/lido-charon-distributed-validator-node v0.4.0-rc2

on GitHub

Summary

This release candidate updates Charon to v2.0.0-rc2 for Gloas, and updates the clients to their Hoodi Glamsterdam releases. It also picks up this week's security fixes: the compromised eMudhra certificate authority, the Go 1.26.10 / 1.27.3 rebuilds, and patched monitoring images.

Update

1. Add the Hoodi builder list to your .env. Copy this line exactly, as it is in .env.sample.hoodi. Every node in your cluster must use exactly the same list, so coordinate with your squad:

   CHARON_BUILDER_URLS=https://hoodi.titanrelay.xyz,https://relay-hoodi.ultrasound.money,https://relay-filtered-hoodi.ultrasound.money,https://hoodi-relay.ethgas.com,https://boost-relay-hoodi.flashbots.net,https://hoodi.aestus.live
   

   These are the Hoodi relays on the Lido MEV-Boost relay allowlist that Charon requests Gloas execution payload bids from directly.

2. Update and restart:

   # Stop the node
   docker compose down
   # Save your local .env / custom overrides
   git stash
   # Update your local copy of this repo
   git pull
   # Checkout this release
   git checkout v0.4.0-rc2
   # Re-apply your stashed changes
   git stash apply
   # Restart the node
   docker compose up -d

The Teku, Prysm and Nimbus validator clients run from images built locally on top of the official ones, so the first docker compose up -d after updating takes a little longer.

Verify your setup

Once your node is running, check that Charon generated the proposer config with your builders:

cat .charon/vc-config/proposer-config.json | jq

The builders must list every relay in your CHARON_BUILDER_URLS, in the same order, and the builder section must be identical on every node in your cluster:

{
  "version": 1,
  "proposer_config": {},
  "default_config": {
    "fee_recipient": "0x... (the Lido Execution Layer Rewards Vault)",
    "gas_limit": "200000000",
    "builder": {
      "min_bid": "0",
      "builder_boost_factor": "100",
      "max_execution_payment": "0",
      "builders": [
        { "url": "https://hoodi.titanrelay.xyz" },
        { "url": "https://relay-hoodi.ultrasound.money" },
        { "url": "https://relay-filtered-hoodi.ultrasound.money" },
        { "url": "https://hoodi-relay.ethgas.com" },
        { "url": "https://boost-relay-hoodi.flashbots.net" },
        { "url": "https://hoodi.aestus.live" }
      ]
    }
  }
}

If a node's builder list differs from its peers', Charon logs Mismatching peer builder configuration, ensure the builder flags are identical on all nodes in the cluster, and the app_peerinfo_builder_config_mismatch metric is 1 for that peer.

What's changed since v0.4.0-rc1

• Reth v2.8.0: schedules the Amsterdam fork on Hoodi. Reth operators must run this release before the fork. It also changes some defaults, see the Reth release notes.
• Lighthouse v8.3.1: a high-priority release that updates trusted certificates after the eMudhra CA compromise.
• lido-dv-exit v0.6.0: backs off Obol API retries instead of polling every slot, builds with Go 1.26.10 and patched HTTP/2 dependencies, and moves to Charon v1.11.2.
• Monitoring: Prometheus v3.15.1, Grafana 13.2.4, Loki 3.7.9, Tempo 2.10.10, Alloy v1.21.1 and cAdvisor v0.60.6, with security fixes and the patched Go releases. node-exporter moves from the deprecated bitnamilegacy/node-exporter image to prom/node-exporter:v1.12.1.

Charon (v2.0.0-rc2) and the other clients are unchanged from v0.4.0-rc1.

Known issues

• Grandine (cl-grandine) can't produce blocks from the Gloas fork. 3.0.0-rc.1 doesn't implement all Gloas beacon API endpoints yet. Use another consensus client on Hoodi until a Gloas-ready Grandine release.
• Lodestar beacon node with the Nimbus validator client: Charon rejects the Nimbus validator client's sync committee contributions (status-im/nimbus-eth2#9172). Attestations and block proposals are unaffected.
• See the Charon v2.0.0-rc2 compatibility matrix for the client combinations tested so far. More will be tested before the fork.

If you already ran v0.4.0-rc1

Most operators can skip this section. It only applies if you started v0.4.0-rc1 before setting CHARON_BUILDER_URLS, or with a different builder list from the one above. The v0.4.0-rc1 notes listed the URLs with a trailing /, and Charon treats https://hoodi.titanrelay.xyz/ and https://hoodi.titanrelay.xyz as different lists.

Charon writes .charon/vc-config/proposer-config.json the first time it starts, and never changes it afterwards. If your proposer-config.json doesn't match the builder list above, set CHARON_BUILDER_URLS exactly as above, then regenerate the file. Replace vc-lodestar with the VC set in your .env (vc-nimbus, vc-prysm or vc-teku):

1. Stop Charon and your validator client: docker compose stop charon vc-lodestar
2. Remove the outdated proposer config: rm .charon/vc-config/proposer-config.json
3. Start Charon first. It generates a new proposer config from your .env: docker compose up -d charon
4. Check the new file (see Verify your setup): cat .charon/vc-config/proposer-config.json | jq
5. Start your validator client. It reads the file on start: docker compose up -d vc-lodestar

If the file and your .env don't match, Charon logs Builder flags differ from the proposer config file.

What's Changed

• images: bump Lighthouse v8.3.1, Reth v2.8.0 and monitoring for rc3 by @OisinKyne in #334

Full Changelog: v0.4.0-rc1...v0.4.0-rc2