doobidoo/mcp-memory-service v8.27.0

v8.27.0 - Hybrid Storage Performance & Real-time Sync

on GitHub

🔄⚡ Hybrid Storage Performance & Real-time Sync

This release brings two major enhancements to hybrid storage: bi-directional sync with live progress updates and 3-5x faster initial sync performance.

🎯 Key Features

Bi-directional Sync with Real-time SSE Progress (PR #229)

• Complete Synchronization: "Sync now" button now performs PULL from Cloudflare + PUSH to Cloudflare
• Problem Solved: MCP-created memories (direct Cloudflare writes) previously required server restart to appear locally
• Live Progress Updates: Real-time SSE events broadcast sync progress every 10 memories
• Enhanced UX:
  • Progress updates: "Syncing: 50/100 (50%)"
  • Completion notifications: "Pulled 50 from Cloudflare, pushed 25 to Cloudflare"
  • Dashboard auto-refreshes when sync completes

3-5x Faster Initial Sync Performance (PR #230)

• Performance Metrics:
  • Before: ~5.5 memories/second (8 minutes for 2,619 memories)
  • After: ~15-30 memories/second (1.5-3 minutes for 2,619 memories)
• Optimizations:
  • Bulk hash loading eliminates 2,619 individual DB queries
  • Parallel processing with 15 concurrent operations
  • Batch sizes increased from 100 to 500 memories (5x fewer API calls)
• Benchmark Results: 78% reduction in Cloudflare API calls (27 → 6 for 2,619 memories)

📊 Combined Impact

Users with large memory databases (1000+ memories) will experience:

• Instant sync visibility: See MCP-created memories immediately after manual sync
• Dramatically faster initial sync: 3-5x performance improvement
• Real-time progress tracking: Know exactly what's happening during sync
• Better UX: Dashboard auto-refreshes with latest data

🔧 Technical Details

Files Modified:

• src/mcp_memory_service/storage/hybrid.py - Bi-directional sync, parallel processing
• src/mcp_memory_service/storage/sqlite_vec.py - Bulk hash loading method
• src/mcp_memory_service/web/api/sync.py - Enhanced sync endpoint
• src/mcp_memory_service/web/sse.py - Progress event broadcasting
• src/mcp_memory_service/web/static/app.js - SSE event listeners

New Configuration:

• MCP_HYBRID_SYNC_OWNER - Control which process handles Cloudflare sync (http/mcp/both)

🛡️ Backward Compatibility

• Zero breaking changes - All existing sync operations enhanced transparently
• Default behavior preserved - "both" sync owner for compatibility
• Graceful fallback - Works with or without Cloudflare connectivity

📚 Documentation

• Full details: CHANGELOG.md
• Performance benchmarks: scripts/benchmarks/benchmark_hybrid_sync.py
• Setup guide: Wiki

🙏 Credits

• PR #229: Bi-directional sync with real-time SSE progress
• PR #230: 3-5x faster sync with bulk operations and parallel processing

Thank you to all contributors and users who provided feedback on hybrid storage performance!

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com