domainaware/parsedmarc 11.0.0

on GitHub

Changes

• Breaking: the output and mailbox integrations are now optional extras, so pip install parsedmarc installs the parsing core and a working core CLI instead of every SDK (#883). A 10.x install pulled in the Elasticsearch, OpenSearch, Kafka, AWS (boto3), Azure, Gmail, and Microsoft Graph client libraries whether or not a deployment used any of them — roughly 1 GB of site-packages against roughly 300 MB without — which is a lot to ask of the motivating case, running parsedmarc as a parsing library on a small mail host. The base install now covers the parsing library plus the CLI reading from files, IMAP, Maildir, and mbox, and writing CSV/JSON, Splunk HEC, webhook, and syslog output; those outputs need only httpx and the standard library, so they deliberately have no extra of their own. Everything else moves behind an extra: elastic (Elasticsearch), opensearch (OpenSearch, including the boto3 SigV4 signer), kafka, s3, gelf, loganalytics (Azure Monitor), msgraph (Microsoft 365 mailboxes), and gmail (Gmail API mailboxes), joining the postgresql extra that already existed. CLI users upgrading from 10.x should switch their upgrade command to pip install -U "parsedmarc[all]" — the new umbrella extra — to keep every integration available; add postgresql to the list (parsedmarc[all,postgresql]) if the PostgreSQL backend is in use. all deliberately excludes postgresql because psycopg's prebuilt binary wheels do not exist for every platform, and pip install parsedmarc[all] must not fail on a platform they do not cover. Users of the prebuilt Docker image (ghcr.io/domainaware/parsedmarc) see no change at all: the image now installs [all,postgresql], so it still bundles every integration. A configuration section whose extra is missing no longer fails with an ImportError traceback at startup; it fails fast with a ConfigurationError naming both the section and the command that fixes it, e.g. The [elasticsearch] configuration section requires the elastic extra: pip install parsedmarc[elastic]. Finally, the never-imported dateparser dependency is dropped in favor of declaring python-dateutil, which parsedmarc.utils actually imports and which used to arrive only transitively through dateparser.
• The CLI now accepts --dns-timeout as an alias of --dns_timeout, which is kept for backward compatibility (public since 6.0.0); --dns-retries already used the hyphenated form.
• Failure and SMTP TLS reports are now sent to Kafka as one message per report, matching the aggregate saver's long-documented per-record behavior. save_failure_reports_to_kafka and save_smtp_tls_reports_to_kafka documented per-record sends in every released version, but the code actually sent the entire report list as a single Kafka message (an unreleased docstring pass in #888 had briefly aligned the wording to the buggy code); a large batch could exceed Kafka's default 1MB message limit, and failure reports in particular carry message samples that make that more likely. Both savers now send/flush one message per report, mirroring save_aggregate_reports_to_kafka's existing per-slice shape. This is consumer-visible: consumers now receive individual report objects on these topics rather than one JSON array per batch.
• dashboard-dev-bootstrap.sh (the contributor dashboard dev-stack bootstrap) now works with Podman as well as Docker — it auto-detects a working container engine (Docker preferred when both are usable) and its Compose implementation (docker compose/docker-compose, podman compose/podman-compose), selectable explicitly with --backend docker|podman or the CONTAINER_BACKEND environment variable. Contributor tooling only; no behavior change for the parsedmarc package itself.

Bug fixes

• [elasticsearch]/[opensearch] number_of_replicas is no longer ignored when number_of_shards is not also set — the parser only read number_of_replicas inside the number_of_shards branch (accidental nesting dating to the 6.4.0-era code), while the documentation lists the two options independently and the client code accepts them independently.
• The CLI now logs a warning for each file path argument that matches no files, instead of silently succeeding with empty results. _expand_file_path_args glob-expands each file_path argument, so a non-existent plain path (e.g. a typo in a cron job) previously vanished as a zero-match glob with no message of any kind; a mailbox-only run that passes no file arguments still logs nothing.