github ridafkih/keeper.sh v2.7.0

latest releases: v2.8.8, v2.8.7, 2.8.7...
7 hours ago

Keeper.sh v2.7.0

This includes a pretty significant set of changes that should improve reliability and performance pretty significantly - particularly with halted syncs requiring a restart. This should no longer be an issue as I've hardened the idempotency of the syncing engine.

Sync Engine Rewrite

  • This includes a complete rewrite of the sync engine to eliminate the immense database backpressure that would occur, and lock up syncing.
  • Replaced the legacy four-job sync infrastructure with two decoupled jobs, the ingest-sources jobs, and the push-destinations job. Each does the respective action on an interval, with the push interval depending on your plan.
  • All event states changes are now computed in memory and flushed in a single database transaction, which should alleviate connection pool backpressure.
  • Operations now occur in chronological order by event start time, and deletes are processed before additions to prevent partial syncs leaving your calendar in a temporary duplicate state.
  • Added a shared Redis-backed sliding window rate limitation (600 requests/min.) for Google, which is shared between runtimes. Reduced to 500 requests per minute to give it a little headroom.
  • Added a per-calendar sync lock which ensures that at most one active sync per calendar can exist, with graceful wind-down on supersession. Partial results are flushed, and not discarded which should make for a more graceful handoff.
  • Syncs now gracefully wind down on timeout (5-minutes) instead of being absolutely smashed by the Promise#race.
  • Mappings are always flushed for successfully pushed events.
  • Google Batch API should now make for much faster sync times for Google. Before, I had implemented sequential and atomic requests in a for of await which meant that, if resolution of the request took the average ~1.5s, syncing 1,000 events would take ~25 minutes. 1,000 events should now take ~2 minutes.
  • Hardened the sync status on the dashboard page.

Code Cleanliness

  • This includes a pretty significant code cleanup, in which 18 packages were consolidated down into 7 core packages.
  • Implemented path aliasing across the board.
  • Adjusted the Dockerfiles, CI, s6 overlay scripts, etc., to accom., the new paths.
  • Fixed turbo cache inputs to include package.json.

Don't miss a new keeper.sh release

NewReleases is sending notifications on new releases.