Substreams Performance (RPC V4, New blocks from last partial)
- RPC V4 protocol with
BlockScopedDatasbatching: MultipleBlockScopedDatamessages are now batched into a singleBlockScopedDatasresponse, reducing gRPC round-trips and message framing overhead during backfill. Clients automatically fall back V4 → V3 → V2 when connecting to older servers, so no flag changes are required. - S2 compression is now the default: Replaces gzip as the default compression algorithm, providing ~3-5x faster compression/decompression with comparable ratios. The client automatically negotiates compression with the server.
- VTProtobuf fast serialization: Both client and server now use vtprotobuf for protobuf marshaling/unmarshaling, providing ~2-3x faster serialization with reduced memory allocations.
- Server-side message buffering: Configurable via
--substreams-tier1-output-buffer-sizeflag (default: 100 blocks) orMESSAGE_BUFFER_MAX_DATA_SIZEenvironment variable (default: 10MB). - Improved Connect/gRPC protocol selection: Server now efficiently routes requests to the appropriate handler based on content-type, improving performance by ~15% for pure gRPC clients.
- New blocks from last partial: "Last partial blocks" are now accepted interchangeably with
newblocks, allowing faster full blocks for requests that do not ask for partial blocks.
Added
- Add
firecore tools substreams logs connections <user_id>command to query Cloud Logging and show Substreams connections for an organization. Correlates incoming requests with stats by trace ID and presents a summary table showing active, closed, and error connections with details like network, source IP, module, duration, and blocks processed.
Removed
- Remove alpha partial blocks support in firehose service (only exposed via substreams)