certtools/intelmq 2.3.1

2.3.1 Bugfix release

on GitHub

Installation documentation:
https://intelmq.readthedocs.io/en/maintenance/user/installation.html
Upgrade documentation:
https://intelmq.readthedocs.io/en/maintenance/user/upgrade.html

Core

• intelmq.lib.utils:
  • log: Handle null value for logging parameter logging_max_size (PR#1786 by Sebastian Wagner, fixes #1778).
• intelmq.lib.pipeline:
  • Amqp._get_queues: Check virtual host when retrieving queue sizes. Fixes output of intelmqctl check for orphaned queues if AMQP is used and the AMQP user has access to more virtual hosts (PR#1830 by Sebastian Wagner, fixes #1746).

Bots

Collectors

• intelmq.bots.collectors.shadowserver.collector_reports_api: Added debug logging to show number of downloaded reports and download size (PR#1826 by Sebastian Wagner, partly addresses #1688 and #1823).

Parsers

• intelmq.bots.parsers.cymru.parser_cap_program:
  • Adapt parser to new upstream format for events of category "bruteforce" (PR#1795 by Sebastian Wagner, fixes 1794).
• intelmq.bots.parsers.shodan.parser:
  • Support nested conversions, improved protocol detection and extended Shodan parser mappings (PR#1821 by Mikk Markus Möll).

Documentation

• Add missing newlines at end of docs/_static/intelmq-manager/*.png.license files (PR#1785 by Sebastian Wagner, fixes #1777).
• Ecosystem: Revise sections on intelmq-cb-mailgen and fody (PR#1792 by Bernhard Reiter).
• intelmq-api: Add documentation about necessary write permission for the session database file (PR#1798 by Birger Schacht, fixes intelmq-api#23).
• FAQ: Section on redis socket permissions: set only minimal necessary permissions (PR#1809 by Sebastian Wagner).
• Add document on hardware requirements (PR#1811 by Sebastian Wagner).
• Feeds: Added Shodan Country Stream (by Sebastian Wagner).

Tests

• Add missing newlines at end of various test input files (PR#1785 by Sebastian Wagner, fixes #1777).
• intelmq.tests.bots.parsers.shodan.test_parser: Add test cases for new code (PR#1821 by Mikk Markus Möll).
• intelmq.tests.lib.test_harmonization.test_datetime_convert: Only run this test in timezone UTC (PR#1825 by Sebastian Wagner).

Tools

• intelmqsetup:
  • Also cover required directory layout and file permissions for intelmq-api (PR#1787 by Sebastian Wagner, fixes #1783).
  • Also cover webserver and sudoers configuration for intelmq-api and intelmq-manger (PR#1805 by Sebastian Wagner, fixes #1803).
• intelmqctl:
  • Do not log an error message if logging to file is explicitly disabled, e.g. in calls from intelmsetup. The error message would not be useful for the user and is not necessary.

Known issues

• Bots started with IntelMQ-API/Manager stop when the webserver is restarted (#952).
• Corrupt dump files when interrupted during writing (#870).
• CSV line recovery forces Windows line endings (#1597).
• intelmqdump: Honor logging_path variable (#1605).
• Timeout error in mail URL fetcher (#1621).
• Shadowserver Parser: Drone feed has (also?) application protocol in type field (mapped to transport protocol) (#1763).