v3.14.0

23 commits since v3.13.1: 13 fix, 8 feat, 1 ci, 1 docs (these notes). Eight feats and no breaking change, so minor.

The headline is optional user accounts (#2128). They are off by default: without a
userManagement block in config.json, nothing in this section changes anything.

Read this before upgrading

These change what a running instance does without being asked.

Limits on unauthenticated endpoints. Requests above them now get an error instead
of an answer:
- GET /api/packets?nodes= takes at most 50 entries and answers 400 above that
  (#2120). 12,000 entries took 1.3 s per request under the store's read lock.
- POST /api/decode and POST /api/packets/observations cap the request body
  (#2119). One 100 MB request raised the process's memory on a test instance.
- GET /api/nodes/{pubkey}/reach runs at most 2 cold scans at once and answers 429
  with Retry-After: 5 beyond that (#2127). Cached answers are not affected.
- The ingestor truncates observer ids, names and other status fields to 128
  characters and IATA codes to 16, and strips control characters (#2123). The
  /neighbors report only accepts 64-hex pubkeys and a scope list of up to 256
  bytes (#2122).
traffic_share_score drops after the upgrade (#2117). The score counted a
transmission once per observation through a relay, so it grew with uptime until many
relays sat at 1.0. It now counts distinct transmissions. Expect lower values; they no
longer drift.
The /api/nodes region filter covers the packet store's window (#2114), the same
window the rest of the UI shows, instead of all database history. A node heard in a
region only before that window no longer matches.
The geo-filter save keeps config.json's file mode (#2126). A 0600 config stayed
0600 only until the first save; now it stays 0600.

Optional user accounts

Turn them on with a userManagement block. Setup and every option are in
docs/user-guide/accounts.md.
Accounts live in a separate users.db; the server still opens the analyzer database
read-only.

Accounts and roles (#2129). Register with an email address, activate through a
mailed link that also sets the password, log in. Roles user and admin. Admins
manage users and use the operator actions (geo-filter, backup, perf reset) without
the shared API key. Mail goes through Brevo, with delivery status.
Settings sync (#2130). A logged-in user's own nodes, favorites, theme,
customizer settings and filters follow them to every device. Private channel keys
and decrypted messages are never synced.
Admin area (#2138). #/admin with an overview of what needs attention (stuck
activations, bouncing mail, password guessing, a dropped MQTT source), the user
table, and an audit log that now records logins.
Hashtag channel proposals (#2139, closes #2092). Logged-in users propose a
channel; after an admin approves it, the ingestor decrypts it without a restart and
it is listed for everyone. Opt in with userManagement.channelProposals. Every
approved key is tried on each GRP_TXT no key opens: 204 to 216 µs per packet with
320 keys, 272 to 320 µs with 128 approved keys added.
Mail notifications for watched nodes (#2140). "Notify me" on a node page sends
one mail when that node goes offline, comes back or reports a low battery, using the
thresholds the node page uses. Admins can add new foreign nodes and observers going
offline. Bundled per user, 20 mails per user and 100 per instance per rolling 24
hours, one-click unsubscribe. Opt in with userManagement.notifications.
Data export and users.db backup (#2141). "Download my data" gives a user one
JSON file with everything stored about their account, credentials excluded. The
server keeps daily users.db snapshots (7 by default) and admins can download one.

Fixes

Regional /api/nodes queries no longer exhaust the database pool (#2114, closes
#2101). A regional node count took 154.8 s on a 10.3 GB database and tied up the
readers. The region set now comes from the packet store: 37 ms uncached in the
benchmark, then cached for 30 s. A follow-up matches observers by id, so a changed
observer region applies at once (#2115).
Escaping: observer IATA, name and id and node names in eight render sinks
(#2118), and the route string on the unknown-route page (#2124).
CDN scripts are pinned with integrity hashes (#2121): chart.js@4.5.1,
leaflet.heat@0.2.0, swagger-ui-dist@5.33.1.

Interface

Path hash size on each channel message (#2089): 1-byte, 2-bytes or 3-bytes
before the scope chip.
RF noise-floor layer on the Mobile RX coverage page (#2113), for instances with
clientRfSamples enabled.
Node details explain the packet and observation counts (#2132, closes #2131).

CI

Tests and the image build run side by side (#2135, closes #2134). Only the GHCR
push waits for all of them, and E2E runs in 3 shards.

Issues closed

#2092 Feature proposal: user-suggested shared hashtag channels with admin approval and revoke
#2101 bug: Regional /api/nodes queries exhaust the SQLite connection pool
#2128 No user accounts: settings are tied to one browser and operator actions to a shared API key
#2131 ui(nodes): explain transmission and observation counts in node details
#2134 CI takes ~30 min because every job waits for the previous one

#2092 and #2128 were closed by hand after their pull requests merged; the others
through the pull request.

Contributors

Pull request authors: @efiten (11), @nullrouten0 (9), @dborup (1), @n30nex (1),
@sylr (1).

Issue reporters: @efiten (2), @dborup (1), @mannkind (1), @n30nex (1).

No co-authors other than tooling are credited in the commits.

Upgrade notes

No manual migration step. With user management off, no new file or table is created.
With user management on, users.db is created next to the analyzer database (or at
userManagement.dbPath) and migrated at startup; backups/ is created next to it.
channelProposals and notifications are opt-in under userManagement; the backup
is on by default when user management is on and can be turned off with
userManagement.backup.enabled: false.

What's Changed

feat(coverage): RF noise-floor layer on the Mobile RX coverage page by @efiten in #2113
feat(channels): show the sender's path hash size on each message by @sylr in #2089
fix(nodes): resolve the /api/nodes region filter from the packet store by @efiten in #2114
fix(nodes): match the region node filter on observer ID, not ingest-time IATA by @efiten in #2115
feat: optional user accounts (part A: foundation) by @efiten in #2129
feat: sync a logged-in user's settings across devices (part B) by @efiten in #2130
fix(store): index a transmission once per relay key in byPathHop by @dborup in #2117
fix(ui): escape the route string on the unknown-route page by @nullrouten0 in #2124
fix(server): keep config.json's file mode when saving the geo-filter by @nullrouten0 in #2126
fix(ui): pin CDN script versions and add integrity hashes by @nullrouten0 in #2121
fix(api): cap the nodes= list on /api/packets at 50 entries by @nullrouten0 in #2120
fix(ingestor): cap observer-supplied string lengths by @nullrouten0 in #2123
fix(ingestor): bound the unauthenticated /neighbors report by @nullrouten0 in #2122
fix(api): cap request bodies on /api/decode and /api/packets/observations by @nullrouten0 in #2119
fix(api): cap concurrent cold reach scans at 2, answer 429 beyond that by @nullrouten0 in #2127
ci: run E2E, Go tests and the image build side by side by @efiten in #2135
feat: admin dashboard with overview and audit log (user management part C) by @efiten in #2138
feat: propose and approve hashtag channels (user management part D) by @efiten in #2139
feat: mail notifications for watched nodes (user management part E) by @efiten in #2140
feat: account data export and users.db backup by @efiten in #2141
fix(nodes): explain packet and observation counts by @n30nex in #2132
fix(ui): escape observer iata, name and id, and node names, in eight render sinks by @nullrouten0 in #2118
docs: release notes for v3.14.0 by @efiten in #2143

Full Changelog: v3.13.1...v3.14.0

Kpa-clawbot/CoreScope v3.14.0 on GitHub