github Galxe/gravity-sdk v1.6.2
Gravity v1.6.2

latest releases: v1.10.1, v1.10.0, v1.9.3...
4 months ago

Gravity v1.6.2

Release: v1.6.2

Full Changelog: v1.6.1...v1.6.2

Related pinned components:

  • gravity-reth: rev acc45884 (fix(rocksdb): flush batch data into storage to make sure stage is completed, gravity-reth#340)

A short two-PR patch release. Both items target single, narrowly-scoped reliability issues observed in production.

Highlights

  • RocksDB durability fix for low-traffic state-trie databases: state-trie commit() previously returned without an fdatasync(), so account_db / storage_db could go 2,000+ blocks without syncing — a power-failure window that would cause large trie data loss and a state-root mismatch panic during consensus re-execution on restart.
  • Sentinel 100% CPU busy-spin fixed. The linemux 0.3.0 upstream bug (jmagnuson/linemux#57) silently spun six sentinel processes at 100% CPU for 21 days on production with all log-based alerting disabled. Switches to a Galxe fork with a 6-line patch.

What's Changed

Bug Fixes

  • fix(sentinel): switch to patched linemux fork to stop 100% CPU busy spin by @nekomoto911 in #728
    • Sentinel's linemux dep is bumped to a Galxe fork (Galxe/linemux@0194e022) that fixes the upstream busy-spin when watched_files becomes empty. The patch is six lines (unwrap_or_continue! chain → explicit match); filed upstream as jmagnuson/linemux#76 (issue) + jmagnuson/linemux#77 (PR), but we don't gate prod recovery on upstream review cadence.
    • Defense in depth: Reader::next_line now handles Ok(None) with exit(2) so systemd restarts the process if the patched linemux ever does report an exhausted stream. Ships a systemd unit template at bin/sentinel/systemd/sentinel.service (Restart=always) so the exit→restart path is real.
    • Regression test in the fork (regression_no_busy_spin_on_drained_watched_files) constructs the exact production state (Inner::readers non-empty + MuxedEvents::watched_files empty + StreamState::Events) and asserts next_line() resolves to Ok(None) within 500 ms. Reverting the patch causes the test to hang (busy-spin), confirming the test catches a regression.

Dependencies

  • chore(deps): bump gravity-reth to acc45884 (rocksdb durability fix) by @nekomoto911 in #730
    • Bumps gravity-reth 41c0b709 → acc45884. Range is exactly one commit ahead (ahead 1, behind 0); this bump introduces gravity-reth#340 and nothing else.

gravity-reth changes since v1.6.1

The greth pin moved from 41c0b709 to acc45884 — exactly one commit:

  • #340 fix(rocksdb): flush batch data into storage to make sure stage is completed by @AshinGau
    • provider_rw.commit() called rocksdb::DB::write() without fdatasync(). With wal_bytes_per_sync=4MB, the low-traffic account_db / storage_db could go 2,000+ blocks without syncing, risking large trie data loss on power failure and a state-root mismatch panic during consensus re-execution after a crash.
    • The fix:
      1. All write() calls in commit_view() use write_opt(sync=true) — every commit() is fdatasync-durable before returning.
      2. Disable pipelined writes and drop wal_bytes_per_sync (redundant with per-commit sync).
      3. Recovery always re-checks every stage checkpoint (no early return), repairing only stages that are behind.

Don't miss a new gravity-sdk release

NewReleases is sending notifications on new releases.