github streamingfast/firehose-ethereum v2.11.12

latest releases: v2.15.9, v2.15.8, v2.15.7...
8 months ago

[!WARNING] This version contains a thread leak, leading to eventual higher memory usage, bump to v2.11.13

Substreams improvements v1.15.8

Rework the execout File read/write to improve memory efficiency:

  • This reduces the RAM usage necessary to read and stream data to the user on tier1,
    as well as to read the existing execouts on tier2 jobs (in multi-stage scenario)

  • The cached execouts need to be rewritten to take advantage of this, since their data is currently not ordered:
    the system will automatically load and rewrite existing execout when they are used.

  • Code changes include:

    • new FileReader / FileWriter that "read as you go" or "write as you go"
    • No more 'KV' map attached to the File
    • Split the IndexWriter away from its dependencies on execoutMappers.
    • Clock distributor now also reads "as you go", using a small "one-block-cache"
  • Removed SUBSTREAMS_OUTPUT_SIZE_LIMIT_PER_SEGMENT env var (since this is not a RAM issue anymore)

  • Add uncompressed_egress_bytes field to substreams request stats log message

Poller

  • add --headers flag to fireeth tools poller to allow auhenticated calls to ETH_RPC providers
  • add --allow-empty-receipts-on-block-0 bool flag to work with tron-evm-mainnet
  • add --parallel-workers int flag to allow increasing from the default (which is now 20 instead of 10)

Various

  • (dstore) Add storageClass query parameter for s3:// urls on stores (@fschoell)
  • Update the firehose-beacon proto to include the new Electra spec in the 'well-known' protobuf definitions (@fschoell)
  • Use The Graph's Network Registry to recognize chains by genesis blocks and fill the 'advertise' server on substreams/firehose

Don't miss a new firehose-ethereum release

NewReleases is sending notifications on new releases.