jmasarweh/UniFi-Insights-Plus v3.7.2

v3.7.2: Disk Reads Fix and Identity Hardening

on GitHub

For full documentation and setup guides, visit insightsplus.dev/docs.

Storage and disk reads

• Periodic full-table reads are gone. Every 15 minutes the receiver ran an exact COUNT(*) over the whole logs table just to write a debug line, and the log-based WAN detection scanned every firewall row. On a 16 million row install that was about 20 GB of reads per run and hundreds of gigabytes per day of SSD wear. The stats job now runs only when LOG_LEVEL=DEBUG and uses the PostgreSQL catalog estimate; WAN and gateway detection looks only at the 7 days before the newest firewall row through the existing index, runs every 6 hours instead of every 15 minutes, and only when the UniFi integration is not the source of identity. Measured idle reads on the same install: 3.66 million heap blocks per 30 minutes before, 2 after. Reported by @culdev and @adrienjp (#103). The approach follows the withdrawn PRs #105 and #139 by @leto1210.
• AbuseIPDB queue pull waits for budget. The threat backfill no longer reads and locks due queue rows when the daily quota is exhausted.
• Setup status no longer counts rows. The unauthenticated /api/setup/status endpoint used an exact firewall row count; it now answers with a boolean has_logs from a single index lookup. The logs_count field is replaced by has_logs.

Network identity (log detection installs)

• A quiet gateway no longer erases learned identity. Detected WAN IPs are merged into the persisted map instead of replacing it: a second WAN with no traffic in the window keeps its address, interfaces removed from the WAN list are pruned, and nothing is ever written from an empty observation. Gateway IPs are a union that never drops an address automatically.
• Learned identity reaches the running receiver immediately. After a detection run that changes anything, the parser and the enrichment exclusions reload without waiting for a restart or an unrelated settings save. A relevant config change (UniFi integration toggled, or a configured WAN interface with no learned IP) requests a refresh right away instead of at the next 6 hour run.
• The setup wizard merges instead of overwriting, and skips log detection while the UniFi integration is effectively enabled; its response now reports identity_source.

Robustness

• The scheduler thread survives transient database errors in the identity refresh and in the retention job rebuild, so retention, blacklist and auth cleanup keep running.
• Each SIGUSR2 reload step (parsers, UniFi, Pi-hole, enrichment, disabled log types) runs on its own, so one failing step no longer skips the others.
• WAN_REFRESH_INTERVAL_MINUTES (new, default 360) is validated: blank or invalid values fall back to the default with a warning instead of a crash or a scheduler stall.

What this release does not change

Reads caused by ingestion, UI and API queries, and the nightly retention cleanup are unchanged. Next on the storage and quota side, each tracked in its own issue: reclaiming space after retention cleanup without a manual VACUUM (#102), AbuseIPDB free-tier quota handling and a daily lookup cap (#108, #132), and a configurable database query timeout (#82).