Summary

This pre-release is optional for now, but strongly recommended.

In particular, it’ll become required for operators to upgrade to v0.8.0 or higher before 2024/09/15 on testnet.

Two changes are needed to complete the upgrade:

• Upgrade image to v0.8.0 or higher
• Set the node environment variable “NODE_SRS_LOAD=8388608”

Both are included in release https://github.com/Layr-Labs/eigenda-operator-setup/releases/tag/v0.8.0-rc.1

Notable features

• EigenDA users - Max Blob size raised from 2MiB to 4MiB
  • Not enabled for custom quorums with less than 8 operators
• Operators - Gnark chunk encoding to reduce data size by about 30%
  • Not enabled for now
  • Scheduled to enable after operators upgrade to v0.8.0 or later version
• NodeInfo API at Node: Operators will start making the Node software version information available to the Disperser
• Churn approval expiration reduced from 60 mins to 15 mins: Churner can approve a churn request every 15 mins now

Other notable changes

• Fixed a bug in blob state transition in Batcher (reset DISPERSING blobs back to PROCESSING at Batcher start)
• Improved performance logging

Extra details for blob size limit and chunk encoding

Increasing max blob size is a notable feature enabled in the release. This feature allows EigenDA users to send blobs of 4MiB, previously 2MiB. For operators, they will need to update the NODE_SRS_LOAD to support larger blobs. The resource utilization on the node is expected to be almost identical under the same level of traffic.

This pre-release also contains chunk encoding changes that are not enabled for now, but are scheduled to enable at some future release, to give operators ample time to upgrade. When enabled after the release post 2024/09/15, the nodes with versions before v0.8.0 will not be able to parse the chunks from Disperser.

We will make an announcement in that future release when the changes are enabled.

What's Changed

• Simplify the bytes decoding of chunks by @jianoaix in #607
• Pull ejector out of dataapi by @ian-shim in #608
• Update eigenlayer-middleware by @0x0aa0 in #611
• Record args in mock transactor by @ian-shim in #616
• [1/N] Chunk encoding optimization: Support encoding format with gnark by @jianoaix in #617
• fix dataapi docker build by @ian-shim in #618
• Use variable rate interval for blob thoughput metrics based on time interval by @pschork in #621
• Add configuration setting for high-level EigenDA client for returning as soon as the blob has confirmed by @teddyknox in #620
• Log the batch validation latency breakdown by @jianoaix in #619
• [TxnManager] Make pk wallet compatible with broadcast check by @ian-shim in #626
• Log request serialization time by @jianoaix in #625
• [inabox] Set BATCHER_FINALIZATION_BLOCK_DELAY to 0 by @ian-shim in #631
• [4/N] Chunk encoding optimization: Add bundle encoding by @jianoaix in #632
• Adds NodeInfo endpoint to retriever and dispersal services for observability by @pschork in #630
• Refactor cpu prover by @bxue-l2 in #629
• [2/N] Chunk encoding optimization: bench mark the gnark based encoding by @jianoaix in #624
• [3/N] Chunk encoding optimization: Add support of new encoding format… by @jianoaix in #628
• [minibatch] Add feature flag on batcher by @ian-shim in #634
• Ignore project files for the Jetbrains GoLand IDE. by @cody-littley in #639
• Fix fft_fr.IsPowerOfTwo(). (#83) by @cody-littley in #638
• Remove ejector from dataapi by @ian-shim in #633
• [5/N] Chunk encoding optimization: Add support of new encoding at Node by @jianoaix in #636
• Refactor dispatcher timeout by @ian-shim in #641
• Collect Node Info during reachability check if port scan was successful by @pschork in #640
• Add a metric for DB writing throughput by @jianoaix in #644
• Inabox readme improvements by @cody-littley in #643
• Revert "Refactor dispatcher timeout" by @ian-shim in #645
• Minibatch store interface by @ian-shim in #635
• Update node interface to enable minibatching by @ian-shim in #646
• [minibatch] Create minibatch from encoding streamer by @ian-shim in #637
• Dispatcher method to disperse blobs to node via StoreBlobs endpoint by @ian-shim in #648
• Generate inclusion proof with blob index instead of hash by @ian-shim in #649
• Clean up EnableDualQuorums flag by @ian-shim in #656
• Add Minibatcher by @ian-shim in #655
• Fix typos by @nnsW3 in #659
• Code hygiene: return error instead of bool by @jianoaix in #660
• Doc hygiene: api docs, repo readme, codebase structure by @jianoaix in #662
• [dispatcher] Add method to send AttestBatch request to nodes by @ian-shim in #664
• [minibatcher] Fullbatch state management by @ian-shim in #665
• Minibatch dynamodb client by @pschork in #652
• [6/N] Chunk encoding optimization: Disperser/Retriever support for new chunk encoding by @jianoaix in #650
• Average blob throughput over 9m window by @pschork in #667
• Enable shadow PutItem blob metadata requests to separate dynamo table for minibatch preprod testing by @pschork in #669
• Added metrics utility for generator. by @cody-littley in #673
• Added test utilities. by @cody-littley in #672
• add configurable max blob size by @bxue-l2 in #675
• Log blob size on request by @ian-shim in #678
• [batcher] Recover blob state by @ian-shim in #657
• add grpc recv buffer by @bxue-l2 in #679
• Configuration for new generator functionality. by @cody-littley in #674
• Downgrade encoder error logs by @ian-shim in #682
• [7/N] Chunk encoding optimization: Node support for mixed encoding by @jianoaix in #654
• Added blob table data structure. by @cody-littley in #677
• Keep batch state in Minibatcher by @ian-shim in #668
• Code hygiene: more properly name the chunk encoding format field by @jianoaix in #661
• Fix the operator stake share basis point calculation by @jianoaix in #658
• [node] Add StoreBlobs endpoint for minibatch streaming by @ian-shim in #671
• Add BatchConfirmer by @ian-shim in #670
• Add batch read operation in dynamodb client by @ian-shim in #684
• Adjust the churn approval interval by @jianoaix in #651
• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696

New Contributors

• @cody-littley made their first contribution in #639
• @nnsW3 made their first contribution in #659

Full Changelog: v0.7.4...v0.8.0-rc.1