Release notes

Major highlights

• Significant storage reduction

  ⚠️
  Downgrading from this version to an earlier one is not possible because of the new database format.
  Resync is not required to update to this version. However, you will get the full benefits of disk space optimization only after fresh sync.

• Faster sync due to the new auto-pivot approach

Details

• Storage reduction

  	v1.18.0		v1.19.0
  		Ancient barriers		Ancient barriers
  State	166 GB	170 GB	161 GB	161 GB
  Receipts	477 GB	269 GB	152 GB	104 GB
  Blocks	334 GB	222 GB	334 GB	222 GB
  Other	...	...	...	...
  Total	965 GB	678 GB	662 GB	504 GB

• Receipts DB size reduction

  • Significantly reduced database size by using a different encoding.
    You can’t downgrade from this version without a full db drop and resync. To get the full benefit of the DB size reduction a resync is needed as it will only apply to new receipts. A node will still work fine on 1.19 version without resyncing.
    You can also call the RPC method debug_migrateReceipts(20000000) to rewrite receipts but it only reduces receipt size partially and a resync tends to be faster.
  • Significantly reduced receipt database size by limiting transaction lookup via transaction hash to past year only (similar to Geth). If you need an older lookup, you may keep the old transaction hash index via --Receipts.TxLookupLimit 0.

• Reduced state DB size by about 5% by not storing commonly occurring patterns.

• Lowered memory consumption. On the Mainnet, reduces memory usage was reduced by about 1 GB after resync or full pruning.

• Faster sync due to auto-pivot

  From initialization to snap sync
  v1.17.4	1h 7m 30s
  v1.18.0	28m 22s
  v1.19.0 without auto-pivot	1m 30s
  v1.19.0 with auto-pivot	10s

  State sync will start almost immediately after starting Nethermind. The pivot block will be updated to the one close to the chain head based on the message from the consensus layer. Thus, there’s no need to download a significant amount of the newest blocks before starting state sync.
  By default, auto-pivot functionality in this version will wait for ~15 minutes for CL to send us FCU based on which Pivot will be selected (in the future will be extended). If for any reason FCU will not arrive at Nethermind in that time(no checkpoint sync on the CL side, issue with CL configuration, etc.) you can increase this value by adding the flag --Sync.MaxAttemptsToUpdatePivot=1800 (900 is the default value - 1 attempt = ~1 second, so for 1800 it will try to update the pivot for a total of 30 minutes).

Changelog

Cancun

• Add JSON-RPC endpoints for EIP-4844, needed to exchange blobs with the consensus layer by @flcl42 in #5558
• Refactor transaction broadcasting for a needs of EIP-4844 by @marcindsobczak in #5485 #5619
• Change blob transaction type value to 3 by @flcl42 in #5597

Metrics

• Prometheus metrics improvements by @deffrian in #5570
• Add tags to every metric by @deffrian in #5625
• Report metrics for DB memory size by @benaadams in #5636

Bug fixes and stability

• Health Checks now will give more useful information about the current syncing status, possible problems during the sync process, and better recognition if a node is healthy by @deffrian in #5630
• Allow specifying concurrent new connection count and connect timeout by @asdacap in #5676
• Fix/transactionhash not threadsafe by @asdacap in #5634
• Use Environment.TickCount64 rather than StopWatch.GetTimestamp by @benaadams in #5575
• Zero total difficulty in FindBlock by @deffrian in #5581
• Modernise C# by @benaadams in #5607
• Set pivot to null if snap and fast sync are disabled by @deffrian in #5454
• Fix negative blockNumber exception by @deffrian in #5609
• Shouldn't Wait() on timer threads by @benaadams in #5497
• Allow specifying local ip. And use any by default instead of loopback. by @asdacap in #5635
• Add config for eth_getLogs max block depth by @asdacap in #5652
• Fix eth capabilities for archive nodes by @marcindsobczak in #5655
• Fix misplaced arguments by @deffrian in #5668

Logging

• Revise RocksDB loading failure message by @rubo in #5568
• Enr log Warn -> Debug by @MarekM25 in #5623
• Align only in peer report by @deffrian in #5667

Performance

• Perf/eth getlogs with compact encoding by @asdacap in #5569
• Make sure compact receipt does not get registered as global receipt decoder. by @asdacap in #5586
• Return a couple additional ArrayPoolLists by @benaadams in #5599
• Make some structs readonly by @benaadams in #5604
• Use direct properties rather than Linq by @benaadams in #5605
• Feature/low priority write by @asdacap in #5588
• Refactor HeaderDecoder by @deffrian in #5488
• Fix/Rocksdb cache not set by @asdacap in #5578
• Rename WriteBytesPerSec to BytesPerSec and add direct io flag. by @asdacap in #5641
• Compressing db by @Scooletz in #5498
• Reduce SyncStatus lock contention by @LukaszRozmej in #5647
• Add option to disable low priority writes for full pruning by @asdacap in #5663
• Receipt key with block number by @asdacap in #5651
• Feature/readahead flag by @asdacap in #5645
• Allow configuration for db compression by @asdacap in #5669
• One that touches keccak.bytes by @asdacap in #5675
• Update pivot basing on message from CL by @marcindsobczak in #5256

Other changes

• Remove Nethermind.Native project by @rubo in #5615
• Add CODEOWNERS by @rubo in #5653
• Update submodules by @benaadams in #5611
• Add NUnit Analyzers by @rubo in #5638
• Fix Overseer path for .NET version update and config state by @benaadams in #561
• Removed never called private methods by @benaadams in #5603
• Replace Nethermind.Secp256k1 with a NuGet package by @rubo in #5610
• Refactor and remove obsolete stuff by @rubo in #5614
• Shanghai/Merge Ethereum test fix by @deffrian in #5318
• Boost and fix tests by @rubo in #5656
• Hide from doc by @asdacap in #5654
• temp fix flaky tests by @smartprogrammer93 in #564
• Replace Nethermind.Secp256k1 with a NuGet package by @rubo in #5610
• Replace System.IO.Abstractions with TestableIO.System.IO.Abstractions.Wrappers package by @rubo in #5612
• Replace Environment.Exit with ProcessExitSource and cancelation token by @LukaszRozmej in #5624
• Enable nullability in Api by @deffrian in #5631
• Adding more sepolia bootnodes by @parithosh in #5567

New Contributors

• @parithosh made their first contribution in #5567

Full Changelog: 1.18.0...1.19.0-rc