certtools/intelmq 2.1.1

on GitHub

Install documentation:
https://github.com/certtools/intelmq/blob/2.1.1/docs/INSTALL.md
Upgrade documentation:
https://github.com/certtools/intelmq/blob/2.1.1/docs/UPGRADING.md

Configuration

• Default configuration:
  • Remove discontinued feed "Feodo Tracker Domains" from default configuration.
  • Add "Feodo Tracker Browse" feed to default configuration.

Core

• intelmq.lib.pipeline: AMQP: using port 15672 as default (like RabbitMQ's defaults) for the monitoring interface for getting statistical data (intelmqctl_rabbitmq_monitoring_url).
• intelmq.lib.upgrades: Added a generic upgrade function for harmonization, checking of all message types, it's fields and their types.
• intelmq.lib.utils:
  • TimeoutHTTPAdapter: A subclass of requests.adapters.HTTPAdapter with the possibility to set the timeout per adapter.
  • create_request_session_from_bot: Use the TimeoutHTTPAdapter with the user-defined timeout. Previously the timeout was not functional.

Bots

Parsers

• intelmq.bots.parsers.shadowserver.parser: Fix logging message if the parameter feedname is not present.
• intelmq.bots.parsers.shodan.parser: Also add field classification.identifier ('network-scan') in minimal mode.
• intelmq.bots.parsers.spamhaus.parser_cert: Add support for category 'misc'.
• intelmq.bots.parsers.cymru.parser_cap_program:
  • Add support for phishing events without URL.
  • Add support for protocols >= 143 (unassigned, experiments, testing, reserved), saving the number to extra, as the data would be bogus.
• intelmq.bots.parsers.microsoft.parser_bingmurls:
  • Save the Tags data as source.geolocation.cc.

Experts

• intelmq.bots.experts.modify.expert: Fix bug with setting non-string values (#1460).

Outputs

• intelmq.bots.outputs.smtp:
  • Allow non-existent field in text formatting by using a default value None instead of throwing errors.
  • Fix Authentication (#1464).
  • Fix sending to multiple recipients (#1464).

Documentation

• Feeds:
  • Fix configuration of Feodo Tracker Browse feed.
• Bots:
  • Sieve expert: Document behavior of != with lists.

Tests

• Adaption and extension of the test cases to the changes.

Tools

• intelmq.bin.intelmqctl:
  • check: Check if running the upgrade function for harmonization is necessary.
  • upgrade-config: Run the upgrade function for harmonization.
  • intelmqctl restart did throw an error as the message for restarting was not defined (#1465).

Known issues

• MongoDB authentication: compatibility on different MongoDB and pymongo versions (#1439)
• ctl: shell colorizations are logged (#1436)
• http stream collector: retry on regular connection problems? (#1435)
• tests: capture logging with context manager (#1342)
• Bots started with IntelMQ-Manager stop when the webserver is restarted. (#952)
• n6 parser: mapping is modified within each run (#905)
• reverse DNS: Only first record is used (#877)
• Corrupt dump files when interrupted during writing (#870)