ar-io/ar-io-node r51

Release 51

on GitHub

This is a recommended release that introduces significant enhancements to
metrics, observer capabilities, gateway filtering, and performance. The release
includes metrics improvements with release tracking, major observer enhancements
with offset observation capabilities, enhanced trusted gateway filtering to
prevent caching issues, and optimized chunk retrieval performance.

Added

• Metrics Enhancement: Added release number as default label to all
  Prometheus metrics, enabling filtering and comparison across releases
• Enhanced Data Stream Metrics: Added comprehensive byte tracking with
  getDataStreamBytesTotal counter and getDataStreamSizeHistogram with 4
  buckets (100KB, 1MB, 10MB, 100MB)
• Peer Metrics: Added metrics for preferred peers and peer types, tracking
  "preferred" vs regular "peer" sources and "bucket" vs "general" peer selection
• Observer Offset Observation: Added complete V1 implementation of offset
  observation with cryptographic chunk validation using arweave.js validatePath()
  for enhanced data integrity verification. This feature is currently in testing
  phase and will be gradually enabled across the network
• Observer Gateway Sampling: Added configurable gateway sampling for offset
  observations with OFFSET_OBSERVATION_SAMPLE_RATE (default 5%)
• Observer Reference Gateway Comparison: Added chunk availability comparison
  against reference gateway to identify gateway-specific vs network-wide issues
• Observer Configuration Controls: Added OFFSET_OBSERVATION_ENFORCEMENT_ENABLED
  to control whether offset failures cause gateway assessment failures (default: false)
• Trusted Gateway Filtering: Added comprehensive IP and origin filtering for
  trusted gateways with support for CIDR blocks and X-Real-IP header to prevent
  gateway loops and unexpected caching behavior
• Chunk Offset Sampling Tool: Added gateway health monitoring tool for chunk
  offset sampling (see tools/sample-chunk-offsets)
• Storage Partition Converter: Added Arweave storage partition to height
  range converter script (see tools/arweave-partitions-to-heights)

Changed

• Request Type Labels: Simplified metric request_type labels to 'full'
  (complete data) and 'range' (partial data) for consistency
• Peer Management: Refactored peer management architecture with extracted
  ArweavePeerManager from ArweaveCompositeClient
• Cache Management: Improved cache handling with proper timer cleanup in
  NodeCache
• Chunk Retrieval Optimization: Optimized chunk retrieval to use single
  peer selection per request, reducing overhead
• Offset-Aware Architecture: Implemented TxOffsetSource architecture for
  more efficient chunk retrieval with sync bucket support

Fixed

• Root Transaction Detection:
  • Enhanced logic to prevent incorrect root detection for self-referencing
    transactions
  • Added early exit for self-referencing root transactions
• Data Cache: Fixed data cache to respect SKIP_DATA_CACHE setting and skip
  writes when disabled

Docker Images

• ar-io-core: ghcr.io/ar-io/ar-io-core:fda43c67f9b2ffdd359c6d6109c79cec5d2da7fb
• ar-io-envoy: ghcr.io/ar-io/ar-io-envoy:159d6467108122a3413c5ab45150d334dc9fb78f
• ar-io-observer: ghcr.io/ar-io/ar-io-observer:dd0501fd22a459c0ed22e3f0119c18c30ad2c96e
• ar-io-clickhouse-auto-import: ghcr.io/ar-io/ar-io-clickhouse-auto-import:71bbc13161b69bc28c501d09f586513073d550fe
• ar-io-litestream: ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8
• ao-cu: ghcr.io/permaweb/ao-cu:08436a88233f0247f3eb35979dd55163fd51a153