swiss-armed-forces/cyber-command/cea/loom 1.0.1

on GitLab

Performance Fix (Critical)

This release addresses a critical performance issue where the API could overwhelm Redis under load,
causing significant degradation to ongoing indexing operations.

Root cause: The API was using Redis KEYS scans to track paused queue state and manage the
cache. KEYS is a blocking O(N) operation that scans the entire keyspace — under heavy API usage
(e.g. frequent queue stats polling), this could starve Redis and slow or stall all workers depending
on it.

What was fixed:

• Paused queue state is now stored in a Redis Set (paused_queues_index) using
  SADD/SREM/SISMEMBER/SMEMBERS — O(1) operations — instead of individual keys scanned with
  KEYS
  
• All remaining redis.keys() calls in cache statistics and cache shrinking replaced with
  non-blocking scan_iter
  
• The flush_cache periodic task now uses scan_iter with pipeline batching (flushing in chunks of
  200) instead of a single blocking KEYS scan
  
• Frontend queue statistics poll interval increased from 5s to 10s to further reduce Redis pressure
  

Bug Fixes

• fix(celery): Fixed two related RabbitMQ routing bugs: the ae-loom alternate exchange is
  switched from topic to fanout so unroutable messages with dotted routing keys are no longer
  silently dropped; removed loom:unroutable from task queues/routes to prevent a wildcard binding
  that caused delayed-retry messages to leak into the unroutable queue
  

New Features

• feat(api): New /v1/celery-inspect REST API for fine-grained control over Celery task
  execution, including per-task and per-task-group pause/resume controls and a system-wide throttle
  toggle
  
• feat(api): New /v1/queues endpoint returning raw per-queue message counts (the previous
  overall stats endpoint moved to /v1/queues/stats)
  
• Frontend status indicator now separately renders paused queues and running tasks, with a pause
  icon when queues are paused but no tasks are running