Summary

This pre-release is optional for users and operators to upgrade.

Notable changes

• Performance improvement: Zero serialization of chunks after being encoded in the dispersal flow, eliminating 20s to minutes overhead on dispersal latency
• Performance observability: Added metrics for blob age, blob encoding latency and system throughput
• Robustness fixes:
  • Added blob state transition invariant checking to make sure blob state changes are correct
  • Added blob state recovery for handling failed blobs stuck in DISPERSING state

What's Changed

• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696
• Add BlobMinibatchMapping in minibatch store by @ian-shim in #683
• Create blob minibatch mappings by @ian-shim in #687
• [BatchConfirmer] Apply state transitions for full batches by @ian-shim in #691
• [1/N][zero serialization] Create data structure for passing encode chunks by @jianoaix in #690
• Add an API to fetch blobs from a given batch header hash by @dmanc in #688
• Update README.md by @Ocheretovich in #694
• [node] AttestBatch endpoint by @ian-shim in #676
• Split blob writer code out of larger PR. by @cody-littley in #685
• [2/N][zero serialization] Make Batcher operate on chunks without ser/deser by @jianoaix in #700
• use insecure credential by @bxue-l2 in #711
• Simplify minibatch data model by @ian-shim in #707
• [node] Refactor blob expiration key by @ian-shim in #693
• Compress signatures in minibatch response by @ian-shim in #699
• [node] Make GetBlobHeader compatible with minibatches by @ian-shim in #698
• [1/N][GPU encoder] Add benchmarking code and refactor encoding module by @dmanc in #715
• Add GetBulkBlobMetadata method in BlobMetadataStore by @ian-shim in #703
• Perf observability: Add blob age at key stages in blob's lifecycle by @jianoaix in #719
• Perf observability: Encoder data throughput and blob size breakdown by @jianoaix in #716
• Perf observability: Add blob encoding latency at client side by @jianoaix in #722
• Fix stringification of uint8 by @jianoaix in #723
• [node] Make GetChunks compatible with minibatches by @ian-shim in #712
• Send blob headers to all quorums on minibatch dispersal by @ian-shim in #717
• [3/N][zero serialization] Make Encoder produce and pass chunks in desired format by @jianoaix in #735
• [4/N][zero serialization] Add fallback to the encoder client when it cannot tell chunk encoding format by @jianoaix in #738
• More fixes of quorum (uint8) stringification by @jianoaix in #731
• Perf observability: Add attestation and encoding throughput by @jianoaix in #725
• Created blob verifier worker. by @cody-littley in #708
• Add dispersing transistion check by @bxue-l2 in #741
• Recover failed blobs in encoding streamer by @ian-shim in #733
• Fix: Append metadata when constructing batch by @ian-shim in #746
• Minor updates to error logs by @ian-shim in #747
• Created dockerized workflow for building protobufs. by @cody-littley in #734

New Contributors

• @Ocheretovich made their first contribution in #694

Full Changelog: v0.8.2...v0.8.3-rc.0