v3.6.1

This release restores Certificate Transparency monitoring and Data Leak detection on deployments running behind a corporate proxy, and cuts the alert noise and blind spots of Website Monitoring.

Update Procedure

This release includes a database migration.

Run every command below from the deployment/ directory, in this order:

1. Pull the new images: make pull
2. Stop the stack: make down
3. Start the stack on the new images: make up. Wait until the watcher container is healthy.
4. Apply the database migration: make migrate

Configuration to review if you created your deployment/.env from an older .env.example or customized the Compose files:

• Set SEARX_PATH=../Searx/searx in deployment/.env. The previous value (../Searx) made SearxNG generate its own settings.yml, which refuses the JSON API used by Data Leak. If a Searx/settings.yml was generated next to the Searx/searx/ directory, move it away, then recreate the searxng container.
• If your Compose file overrides the certstream image, use 0rickyy0/certstream-server-go:v1.8.2 (new CERTSTREAM_VERSION variable) instead of latest.
• If your Compose file overrides the watcher environment, add certstream to no_proxy.

What's Changed

Certificate Transparency (CertStream)

• Fixed the listener dying silently after about 140 reconnections and keepalive pings never being sent. Added a watchdog that resets a silent connection and logs a CertStream stats line every 5 minutes.
• Fixed the listener never recovering after MySQL dropped its connection.
• Fixed the callback being unable to follow the feed: monitored domains and keywords are now cached for 30 seconds.
• Keywords are now case-insensitive, and certificates without a subject CN are matched on their SAN list.

Corporate Proxy Support

• Fixed the CertStream WebSocket being sent through the corporate proxy (HTTP 403) instead of reaching the internal certstream service: websocket-client reads the lowercase no_proxy first, and only NO_PROXY was patched. The Compose file now also adds certstream to the watcher no_proxy.
• Internal hosts are now detected properly (private IP, localhost, dotless name or NO_PROXY) instead of a string-prefix test that treated the public certstream.calidog.io as internal.

Data Leak

• Fixed Data Leak silently finding nothing when SearxNG refuses the JSON format: the cause is now logged once per run instead of one cryptic line per keyword.
• Added timeouts to SearxNG and Pastebin requests, and fixed a crash on SearxNG answers without results.

Website Monitoring

• Fixed the alert noise on sites behind a rotating address pool (an alert about every 3 hours per site): Watcher now remembers the /16 networks each site was seen in (30 days) and only alerts when a never-seen network appears. A DNS timeout is also no longer read as the records disappearing.
• Fixed "Alerts for" showing nothing for sites without a very recent alert: it now loads the site's own alerts (new ?site=<id> filter on /api/site_monitoring/alert/).
• Fixed one failing site stopping the checks of the following sites, and crashes on invalid domain labels and on pages too short to be fingerprinted.

Deployment & Infrastructure

• Pinned certstream-server-go to v1.8.2 in Docker Compose and the Helm chart (CERTSTREAM_VERSION): the latest tag moved to v1.9/v1.10, which cannot stream behind an HTTP proxy.
• Fixed the SEARX_PATH default in deployment/.env.example (../Searx/searx): the previous value made SearxNG generate its own settings.yml, which refuses the JSON API.

Full Changelog: v3.6.0...v3.6.1