certtools/intelmq 2.1.2

Bugfix release

on GitHub

Install documentation:
https://github.com/certtools/intelmq/blob/2.1.2/docs/INSTALL.md
Upgrade documentation:
https://github.com/certtools/intelmq/blob/2.1.2/docs/UPGRADING.md

Core

• __init__: Resolve absolute path for STATE_FILE_PATH variable (resolves ..).
• intelmq.lib.utils:
  • log: Do not raise an exception if logging to neither file nor syslog is requested.
  • logging StreamHandler: Colorize all warning and error messages red.
  • logging FileHandler: Strip all shell colorizations from the messages (#1436).
• intelmq.lib.message:
  • Message.to_json: Set sort_keys=True to get reproducible results.
  • drop_privileges: Handle situations where the user or group intelmq does not exist.
• intelmq.lib.pipeline:
  • Amqp._send and Amqp._acknowledge: Log traceback in debug mode in case of errors and necessary re-connections.
  • Amqp._acknowledge: Reset delivery tag if acknowledge was successful.

Bots

Collectors

• intelmq.bots.collectors.misp.collector:
  • Add compatibility with current pymisp versions and versions released after January 2020 (PR #1468).

Parsers

• intelmq.bots.parsers.shadowserver.config: Add some missing fields for the feed accessible-rdp (#1463).
• intelmq.bots.parsers.shadowserver.parser:
  • Feed-detection based on file names: The prefixed date is optional now.
  • Feed-detection based on file names: Re-detect feed for every report received (#1493).

Experts

• intelmq.bots.experts.national_cert_contact_certat: Handle empty responses by server (#1467).
• intelmq.bots.experts.maxmind_geoip: The script update-geoip-data now requires a license key as second parameter because of upstream changes (#1484)).

Outputs

• intelmq.bots.outputs.restapi.output: Fix logging of response body if response status code was not ok.

Documentation

• Remove some hardcoded /opt/intelmq/ paths from code comments and program outputs.

Packaging

• debian/rules: Only replace /opt/intelmq/ with LSB-paths in some certain files, not the whole tree, avoiding wrong replacements.
• debian/rules and debian/intelmq.install: Do install the examples configuration directly instead of working around the abandoned examples directory.

Tests

• lib/test_utils: Skip some tests on Python 3.4 because contextlib.redirect_stdout and contextlib.redirect_sterr are not supported on this version.
• Travis: Stop running tests with all optional dependencies on Python 3.4, as more and more libraries are dropping support for it. Tests on the core and code without non-optional requirements are not affected.
• tests.bots.parsers.html_table: Make tests independent of current year.

Tools

• intelmqctl upgrade-config: Fix missing substitution in error message "State file %r is not writable.".

Known issues

• bots trapped in endless loop if decoding of raw message fails (#1494)
• intelmqctl status of processes: need to check bot id too (#1492)
• MongoDB authentication: compatibility on different MongoDB and pymongo versions (#1439)
• ctl: shell colorizations are logged (#1436)
• http stream collector: retry on regular connection problems? (#1435)
• tests: capture logging with context manager (#1342)
• Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952)
• n6 parser: mapping is modified within each run (#905)
• reverse DNS: Only first record is used (#877)
• Corrupt dump files when interrupted during writing (#870)