ObolNetwork/lido-charon-distributed-validator-node v0.2.17 on GitHub

This release introduces Charon v1.8.3. This release is a strongly recommended update for Node Operators sending logs to Obol. (Nodes sending only metrics are unaffected by this patch)

If you are updating from v0.2.14 or newer, (and you have set your ALERT_DISCORD_IDS= environment variable as requested in earlier updates), there are no action items to update to this release. Operators upgrading from a version older than v0.2.14 should note that there are breaking changes to monitoring that require environment variable adjustment for those running v0.2.9 and earlier. Details below:

Note

Recent releases adjusts how metrics are sent to Obol. Since recent updates, you no longer have to keep stashing and unstashing changes on prometheus/prometheus.yml every update. It is now generated at runtime by ./prometheus/run.sh, and the required variables are injected into the file from your .env. If you're updating from an older LCDVN version. The steps you need to take to handle this change are described below.

If you currently have a locally modified prometheus/prometheus.yml, make a backup copy of it outside of the repository before upgrading (just in case). Ensure you record the value of PROM_REMOTE_WRITE_TOKEN from the authorization.credentials section of that file.
This token must now be provided via your .env file instead of being defined in prometheus.yml. The variable PROM_REMOTE_WRITE_TOKEN is present (commented out) in all .env.sample.* files. To configure it, copy the variable to your own .env, uncomment it and set your token, for example:

PROM_REMOTE_WRITE_TOKEN=obolH7d...

We are now adding a dedicated way to add discord IDs to your deployment, allowing you to be @'d directly by our Obol Agent if there is an issue with your node. Add (uncomment) ALERT_DISCORD_IDS= in your .env, and specify one or more (with comma separation) IDs. To get the ID that corresponds to your discord account; Enable developer mode on discord with User Settings > Advanced. Then right click on a user's profile picture or name and select Copy ID to get a unique 18-digit number that represents their account. If you previously changed your CHARON_NICKNAME to your discord ID, you can set that back to a human friendly name for your node.
If you have any other custom modifications to your original prometheus.yml, compare it against prometheus.yml.example, which is now used as the base template for generating the final configuration. Any additional fields or customizations should be added to prometheus.yml.example.
With your environment variables set, and any other modifications ported to the prometheus.yml.example file; run docker compose up -d as normal. Confirm that your metrics are being received on the Obol Grafana. Once you have verified that the new Prometheus setup is working as expected, you can safely delete your backup of the old prometheus.yml.

Important

To maximise compatibility of environment variable interpolation across operating systems, changes have been made between this version and v0.2.3.

Please set (uncomment) the following .env vars if you haven't already, or your charon, mev-boost, and charon-dv-exit-sidecar may not start properly.

(Copying the .env.sample.mainnet file afresh and making any modifications you may have made, might be the least disruptive way to pick up these changes.)

CHARON_BEACON_NODE_ENDPOINTS=http://${CL}:5052
CHARON_EXECUTION_CLIENT_RPC_ENDPOINT=http://${EL}:8545
VE_BEACON_NODE_URL=http://${CL}:5052
VE_EXECUTION_NODE_URL=http://${EL}:8545
LIDO_DV_EXIT_BEACON_NODE_URL=http://${CL}:5052
CLUSTER_NAME="Your cluster name here"
CLUSTER_PEER="Your peer name here"

Important

Users should no longer run docker compose -f docker-compose.yml -f logging.yml -f docker-compose.override.yml to run their cluster, they should instead only run docker compose up -d. Overrides to turn off certain containers (e.g. in the case where you use an external EL/CL rather than the one in this repo) should be done with .env file variables, by setting EL=el-none, CL=cl-none, and MEV=mev-none for example. Read more about client swapping in our docs.

Logging can be enabled by uncommenting the line #MONITORING=${MONITORING:-monitoring},monitoring-log-collector in their .env file, and setting CHARON_LOKI_ADDRESSES to a URI given to you by the Obol team.

To update to this version, please run the following commands:

# Stop the node
docker compose down
# Copy your PROM_REMOTE_WRITE_TOKEN and put it in .env after stashing
git stash
# Update your local copy of this repo
git pull
# Checkout this release
git checkout v0.2.17
# You should no longer need to apply any stashed changes if you followed the instructions above
# Restart the node
docker compose up -d

Note

lido-charon-distributed-validator-node is a repo intended as a deployment guide and is not intended to be the canonical way to deploy a distributed validator.

Operators are encouraged to use this repository to build and maintain their own configurations that work for their individual use case. Please work with your squad to have your cluster exposed to no single point of failure on three or more nodes.

What's Changed

Update to charon v1.8.3 by @OisinKyne in #289

Full Changelog: v0.2.16...v0.2.17