fix(logs): a line with no RouterOS id had no protection against re-insertion (v0.24.4-beta)
Follows #137. The v0.24.1 fix made the count honest and stopped a thousand
round trips per poll, but it assumed the unique index on (device_id, log_id)
would reject lines already stored. For most devices it does. For a device
whose log entries carry no .id it never could: the column was set to NULL,
and PostgreSQL treats NULLs as distinct in a unique index, so every NULL
conflicts with nothing. Those devices re-inserted their entire buffer on
every poll, for ever, and the constraint that was supposed to stop it was
structurally incapable of firing.
A line with no id now gets a key derived from its own content -- timestamp,
topics and message -- so the existing index does the work. The # prefix
keeps it out of the space of real RouterOS ids (* plus hex), so the two
cannot collide, and parseRosId already rejects it, which matters because the
watermark is a max over real ids and must not be poisoned by a synthetic one.
The trade, stated rather than buried: two genuinely separate lines sharing a
timestamp, topic and message collapse into one. RouterOS timestamps are
second-resolution, so that can happen. Losing a repeat of an identical
message is a much smaller loss than growing the table by the whole buffer
every minute without bound.
This was found by reading a comment on #137 I had not read before closing it
-- log output from a device with eleven lines total, reporting eleven new
entries every minute for forty-five minutes. My own fleet cannot reproduce
it: every event here carries a real .id, with no nulls and no duplicates,
which is exactly why the hole was invisible from here. So this closes a
defect demonstrable from the schema; whether it is the whole of that
reporter's problem is still an open question and the issue is reopened to ask.
Verified no regression where ids are healthy: no surrogate keys generated, no
nulls, no duplicates, and counts that vary with real activity.
Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Container image: ghcr.io/2gt-media-group-llc/mikrotik-manager:0.24.4-beta
Documentation: https://2gt-media-group-llc.github.io/mikrotik-manager/