ar-io/ar-io-node r58

Release 58

on GitHub

Release 58 - 2025-11-10

This is a recommended release due to significant improvements in data retrieval efficiency and payment system reliability. This release introduces a new raw binary chunk data endpoint providing ~40% bandwidth savings, comprehensive rate limit balance management APIs, and intelligent OpenTelemetry tail-based sampling for cost-effective observability. The release also includes critical payment validation fixes and enhanced bundler service discovery for improved client integration.

⚠️ EXPERIMENTAL FEATURES: The rate limiter and x402 payment protocol are experimental features subject to change. API endpoints, parameters, behavior, and configuration options (environment variables) may evolve in future releases as these systems continue to be developed. See docs/x402-and-rate-limiting.md for comprehensive documentation.

Added

• Raw Binary Chunk Data Endpoint: New /chunk/<offset>/data endpoint returns raw binary chunk data (application/octet-stream) with metadata in response headers instead of base64url-encoded JSON

  • Provides ~40% bandwidth savings compared to the base64url-encoded /chunk/<offset> endpoint
  • Supports both GET and HEAD requests
  • Returns comprehensive metadata in custom headers
  • Supports ETag-based conditional requests (304 Not Modified)
  • Supports Content-Digest header (RFC 9530) for data integrity verification
  • Rate limited at 256 KiB (raw chunk size) vs. 360 KiB for base64url endpoint, resulting in lower per-chunk fees

• Bundler Service Discovery: The /ar-io/info endpoint now includes a bundlers field for client service discovery

  • Configurable via BUNDLER_URLS environment variable (comma-separated URLs)
  • Defaults to https://turbo.ardrive.io/
  • URLs are validated on startup with descriptive error messages

• Rate Limit Balance Management API: New REST API endpoints for querying and managing rate limit bucket balances

  • GET /ar-io/rate-limit/ip/:ip - Query IP-based rate limit bucket balance
  • POST /ar-io/rate-limit/ip/:ip - Top up IP-based bucket via x402 payment or admin API key
  • GET /ar-io/rate-limit/resource - Query resource-based bucket balance
  • POST /ar-io/rate-limit/resource - Top up resource-based bucket via x402 payment or admin API key
  • Dual authentication: x402 payment protocol (public) or admin API key (private/testing)
  • 10x capacity multiplier applied to x402 payments

• OpenTelemetry Collector with Tail-Based Sampling ⚠️ EXPERIMENTAL: New OTEL Collector sidecar in docker-compose deployments implements intelligent tail-based sampling to reduce telemetry costs by 80-95%

  • Five intelligent sampling policies make decisions after traces complete
  • 100% of traces with errors, slow requests, x402 verified payments, and paid rate limit token usage
  • 1% (configurable) of successful, fast, unpaid requests for baseline metrics
  • Support for multiple telemetry backends (Honeycomb, Grafana Cloud, Datadog, New Relic, Elastic APM)
  • Optional deployment via docker-compose profile (docker compose --profile otel up)

Changed

• Chunk Request Routing: Removed Envoy proxy route for GET /chunk/ requests - all chunk endpoints now route directly to the AR.IO gateway application
• Transaction-Level Merkle Path Support: The /chunk/<offset> endpoint now includes tx_path in JSON responses when available
• Observer: Updated to fcd0f36 - Doubled offset observation sample rate to 2% for improved network robustness
• AR.IO Info Endpoint Structure: The bundlers field in /ar-io/info endpoint response is now nested under services.bundlers
• Rate Limiter Bucket Keys: Standardized bucket key format across Redis and Memory rate limiter implementations
• Browser Paywall for Chunk Requests: Optimized payment flow for chunk endpoints to use direct URL payment instead of redirect endpoint

Fixed

• Payment Processor and x402 Validation: Multiple improvements to payment validation and error handling

  • Prevent infinite redirect loop on payment failure
  • Validate payment type before settlement
  • Prevent silent success when payment settlement cannot grant tokens
  • Validate resource target format before settling payment
  • Validate Host header presence and format before processing payments
  • Use configured capacity multiplier consistently

• Rate Limiter Configuration: Fixed configuration handling for consistent behavior

  • Ensure consistent resource key normalization across Redis and Memory implementations
  • Use configurable capacity multiplier in rate limit routes

Docker Images

All images are tagged with their respective git commit SHAs:

• envoy: ghcr.io/ar-io/ar-io-envoy:f7c5fd8f3ad894e7a9e12162941a252b129c04b1
• core: ghcr.io/ar-io/ar-io-core:2ea9703941aedc28cc15ee8e753764fecfce3a7a
• clickhouse-auto-import: ghcr.io/ar-io/ar-io-clickhouse-auto-import:4512361f3d6bdc0d8a44dd83eb796fd88804a384
• observer: ghcr.io/ar-io/ar-io-observer:fcd0f3642d5755c4b6adfdd4fb298d127dbb0f68
• litestream: ghcr.io/ar-io/ar-io-litestream:be121fc0ae24a9eb7cdb2b92d01f047039b5f5e8

Installation

See the README for installation instructions.

For full details, see the CHANGELOG.