v3.0

Watcher v3.0 marks a significant evolution of the platform, delivering fully modernized infrastructure, expanded threat intelligence capabilities, and a greatly improved user experience. This release introduces centralized data management, AI-driven threat analysis, advanced domain tracking, and a completely redesigned interface for faster, more intuitive operations.

Important: This release includes major database schema changes. Ensure you have a complete backup before upgrading to prevent any data loss.

Update Procedure

Please follow this process:

1. Pull the latest Docker image from the repository

   docker compose pull

2. Apply any database migrations:

   docker compose down
   docker compose run watcher bash
   python manage.py migrate

3. Update your .env file and review the new weekly summary and breaking news settings. By default, they are set as:

   WEEKLY_SUMMARY_DAY=Monday
   WEEKLY_SUMMARY_HOUR=9:30
   BREAKING_NEWS_THRESHOLD=15

   You can adjust these values to match your preferred schedule and thresholds.
4. Rebuild and restart containers:

   docker compose down
   docker compose up -d

What's New

Core Infrastructure & Database

Legitimate Domains Module

• New LegitimateDomain module for tracking company-approved domains
• Supports expiry dates, repurchase status, and contact information
• Dedicated API with search, ordering, and full CRUD operations

Enhanced Domain Tracking

• New fields on Website Monitoring module: registrar, legitimacy, domain_expiry, takedown_request, legal_team and blocking_request
• RDAP alert support for tracking registration changes

Threats Watcher Summary System

• New Summary model supporting weekly summaries and breaking news alerts
• AI-generated content with extracted CVEs, organizations, and threat actors

Centralized Logging System

• Introduction of a new Logger dependency providing a unified, application-wide logging layer
• Consistent log formatting and routing across all modules
• Contextual log levels (debug, info, warning, error, critical) with colorized output in development mode

RDAP & WHOIS Discovery

Comprehensive Discovery System

• RDAPDiscovery class with automatic TLD endpoint detection and fallback mechanisms and WhoisDiscovery class for cases where RDAP data is unavailable
• Scheduled automated lookups for domains lacking registrar information

Smart Domain Updates

• Automatic legitimacy status updates when domains transition between available, disabled, and registered states
• Real-time RDAP/WHOIS alert tracking with registrar and expiry date change notifications

Notification System Enhancements

Enhanced Platform Support

• Improved TheHive integration with intelligent alert/case creation
• Dedicated Slack and Citadel message handlers with application-specific formatting

Frontend & UI Improvements

Theme System

• 5 themes: Darkly, Flatly, Cyborg, Superhero, Brite
• Persistent localStorage for user preferences

Advanced Dashboard Features

• ResizableContainer: User-customizable panel widths across all dashboards
• TableManager: Advanced filtering, sorting, pagination, and saved filter sets
• Breaking News Popup: Real-time floating alerts for critical threats with auto-dismiss
• Weekly Summary Widget: Minimizable/draggable card with typewriter effect and CVE detection

Enhanced Visualizations

• New SiteStats and LegitimateStats components with icon-based metrics
• Domain expiry badges and monitoring status indicators
• Unified ExportModal for MISP and Legitimate Domain conversions

Filter Persistence

• Save and load custom filter configurations per module

Threats Watcher Enhancements

AI-Powered Intelligence

• Integration with Hugging Face transformers for NER and summarization
• Advanced Named Entity Recognition with noise filtering
• BART-based text summarization for threat reports
• Cached pipeline management for optimized performance

Automated Threat Detection

• Breaking news detection with threshold-based automatic alerts
• Weekly intelligence report generation with scheduled cron jobs
• Entity extraction for persons, organizations, locations, and products
• Real-time signal system for rapid threat trending

Breaking News & Weekly Reports

• Automatic detection of rapidly trending keywords
• AI-generated summaries with extracted threat indicators
• Scheduled weekly intelligence reports with threat actor identification

Site Monitoring Improvements

6-Level Legitimacy Classification

• Unknown → Suspicious, not harmful → Suspicious, likely harmful (registered) → Suspicious, likely harmful (available/disabled) → Malicious (registered) → Malicious (available/disabled)
• Automated classification updates based on domain registration status
• Historical legitimacy tracking

Enhanced Monitoring

• Dedicated RDAP/WHOIS alert types for registrar and expiry changes
• RDAP-specific modal views showing date differences
• Direct conversion of monitored sites to legitimate domain tracking
• Improved stats display with action badges

DNS Finder Updates

Smart Domain Handling

• Automatic cleaning of *. prefixes from certificate transparency domains
• Enhanced source tracking for cert transparency vs. dnstwist detection
• Better parent_domain and subdomain tagging in TheHive exports

Data Leak Enhancements

• Grouped notifications for multiple alerts on same keyword
• Improved paste content display with download functionality

Testing & Quality Assurance

Unit Tests (Back-End)

• Total of 109 Django unit tests across all core modules, ensuring coverage of existing and updated back-end functionality.

End-to-End Tests (Front-End)

• Total of 229 Cypress tests, covering all major front-end workflows and new features.
• New: LegitimateDomains.cy.js with 51 tests specifically validating the new Legitimate Domains module.

Bug Fixes

• Observable deduplication in TheHive alerts/cases
• Better comment and observable management in notification handlers
• Better formatting and organization in Data Leak module

Full Changelog: v2.4.1...v3.0