idaholab/Malcolm v26.09.0

Malcolm v26.09.0

on GitHub

Malcolm v26.09.0 adds support for uploading raw gzip/bzip2/xz/lzma-compressed PCAP files; a configurable Suricata rule-update schedule, source list, and startup behavior; improvements to NetBox autopopulation, including explicit public-subnet support and most-specific-prefix selection; and an optional, early integration with MITRE's Operational Technology Knowledge Base (OTKB) — a MITRE project not yet publicly released, expected within the next several months — enriching supported Zeek and Wireshark OT events with normalized protocol, function, and MITRE ATT&CK for ICS context ahead of that public release, to support some internal pre-release testing in cooperation with MITRE. Strelka's file-scanning backend also received a round of bug fixes across several individual scanners, and NetBox, Filebeat, Logstash, and other components have been updated. Several other bug fixes and general improvements are also included. The Raspberry Pi 5 Hedgehog Linux image now uses a downstream kernel that restores working thermal monitoring and fan control.

This release also fixes three security issues: an SSRF vulnerability in the file-upload container's bundled FilePond server allowing internal service reconnaissance, an nginx RBAC bypass via unresolved /./ path segments, and a missing auth gate on the /dashboards/app/netbox/ proxy location.

The Malcolm team would like to thank @sylvesterkaczmarek, @pavanchow, and @scofaild23 for their contributions to this release.

If you are upgrading from an existing Malcolm installation, run ./scripts/status for Malcolm to migrate some settings prior to running ./scripts/configure, ./scripts/start, or other Malcolm control scripts.

v26.08.0...v26.09.0

• ✨ Features and enhancements

  • Add support for uploading raw gzip-, bzip2-, xz-, and lzma-compressed PCAP/PCAPNG files; the upload watcher inspects the decompressed stream with libmagic and routes recognized PCAP content to the PCAP processing directory, while non-PCAP compressed streams and tar-based archives continue through the existing archive path unchanged #1075, #462
  • Overhaul Suricata rule-update configuration: SURICATA_UPDATE_CRON_EXPRESSION makes the periodic update schedule configurable #1078; rule updates can now run once at container startup, bounded by a timeout, when SURICATA_UPDATE_RULES=true #1079; SURICATA_UPDATE_ENABLE_SOURCES allows enabling indexed suricata-update sources (e.g., et/pro) with optional parameters #1080; and SURICATA_UPDATE_SOURCES exposes the existing source-index refresh toggle in config/suricata.env #1084 #723
  • Allow NetBox autopopulation to include explicitly-listed public IP subnets in NETBOX_AUTO_POPULATE_SUBNETS (private-only remains the default when the variable is blank) #1102, #908
  • Prefer the most specific (longest-prefix-match) NetBox prefix when enriching IP prefix fields, preventing parent and child prefixes from both being applied and producing multi-valued source.segment/destination.segment fields #1103, #591
  • Add OPENSEARCH_DEFAULT_DASHBOARD to the standard dashboards-helper.env configuration template, documenting the dashboard IDs for four candidate landing dashboards (Overview, Security Overview, ICS/IoT Security Overview, Severity) #1083
  • Add an optional, early integration with MITRE's Operational Technology Knowledge Base (OTKB) — a MITRE project that has not yet been publicly released, with an anticipated future public open-source release — built ahead of that release to support some internal pre-release testing in cooperation with MITRE. It enriches supported Zeek and Wireshark OT events with normalized protocol, function, classification, citation, procedure, asset, campaign, software, and MITRE ATT&CK for ICS context under a new otkb.* field hierarchy, including an OpenSearch composable template, an "OT Knowledge Base" dashboard, and support for connected, disconnected/air-gapped, and file-seeded-then-API-refreshed deployments. Since OTKB itself is not yet public, this integration is primarily useful to those with access to a pre-release OTKB instance or fixture #1124

• 🛡️ Security Remediation & Hardening

  • Fix an SSRF vulnerability in the file-upload container's bundled FilePond PHP server, which shipped a live FETCH_REMOTE_FILE route allowing an authenticated user to make the upload container fetch arbitrary internal URLs (including ftp://) and retrieve the response, enabling reconnaissance of internal services such as OpenSearch, the Malcolm API, Dashboards, and NetBox; the route is now stripped from the vendored PHP at build time and the upstream branch is pinned to a specific commit GHSA-cvx2-hj33-vc4q #1101
  • Fix an RBAC bypass in nginx's Lua authorization layer where /./ single-dot path segments were not resolved by normalize_uri_for_rbac(), though nginx itself resolves them before location selection; prefixing a role-restricted path with ./ let any authenticated user reach every role-restricted surface, including /upload, /server/php, /htadmin, and /auth, in Keycloak RBAC deployments GHSA-v66f-cwcm-hf73 #1106
  • Fix a missing auth gate and case-sensitivity mismatch on the /dashboards/app/netbox/ proxy location, the same class of defect previously fixed for other locations, where a case-varied request could fall through to the NetBox backend without authentication #1107
  • Update the htadmin container from Debian 11/PHP 7.4 to Debian 13/PHP 8.4, since Debian 11 reached end-of-life and its security repository stopped serving updates; the now-unnecessary PECL mcrypt build tooling and packages were removed in the process #1118

• 🐛 Bug fixes

  • Fix the Raspberry Pi 5 Hedgehog Linux image lacking functional temperature monitoring and four-pin fan control by building and installing a matched Raspberry Pi downstream kernel (rpi-6.18.y) with the corresponding DTBs and overlays, rather than Debian's generic ARM64 kernel; Ethernet, Wi-Fi, Bluetooth, Docker, cgroup v2, and the framebuffer console were also validated on the new kernel #1098
  • Allow Raspberry Pi USB-attached storage devices (e.g., USB SSDs) to be selected for the Hedgehog Linux datastore in os-disk-config.py, while continuing to protect the system disk, exclude MMC/other hot-pluggable media, and preserve existing behavior on non-Raspberry Pi hardware #1122
  • Fix Hedgehog Linux on Raspberry Pi failing to set the UFW reachback ACL at startup (prompting for a sudo password and logging ufw_manager.sh: command not found) due to the invoking command not searching /usr/local/bin #1097
  • Fix several latent Strelka scanner bugs found during full-suite testing with all 76 scanners enabled: unhandled exceptions in ScanX509 (malformed certificates), ScanTranscode (malformed HEIF/AVIF), ScanRtf and ScanTnef (outdated oletools/tnefparse API usage); a TypeError in ScanVba when emitting extracted macros; a broken xar package install that produced a non-functional ScanXar; ScanBase64PE routing to a scanner class that was never implemented; disabled vsto_file/email_file_broad YARA rules in Malcolm's fork; and sporadic 404s fetching config.guess/config.sub during the libxar build #1111
  • Fix Zeek container builds failing with zkg install errors against the FoxIO ja4+ plugin, since a raw Git commit hash is no longer a resolvable zkg package version once newer commits are pushed upstream; the plugin is now pinned to the v1.0.1 release tag, which contains the same commit and preserves the field layout Malcolm depends on #1112, #1109
  • Fix NetBox plugin discovery missing plugins installed with pip install --target (invisible to pip list --local), and fix an AttributeError: module 'ast' has no attribute 'Str' crash on Python 3.12+ once such a plugin was found and needed to be added to plugins.py #1125
  • Fix NetBox DHCP hostname autopopulation using the unnormalized [zeek][dhcp][assigned_addr] field instead of the normalized [zeek][dhcp][assigned_ip] field the enrichment logic actually checks #1090
  • Fix the Zeek IANA/ports-and-protocols generator not clearing a service value of unknown due to checking the wrong loop variable #1089
  • Fix the Zeek IANA/ports-and-protocols generator allowing an incomplete Drahgkar source row to reuse values from the previous row or raise UnboundLocalError #1091
  • Fix a shadowed storage configuration key constant and a duplicated https:// in the Arkime ILM help link in the installer #1092
  • Fix NetBox plugin class discovery failing on qualified PluginConfig base classes (e.g., netbox.plugins.PluginConfig), which are ast.Attribute nodes rather than the ast.Name nodes the AST check expected #1093
  • Fix missing commas in zeek_intel_from_threat_feed.py's CLI help text that caused two adjacent help entries to be concatenated #1094

• ✅ Component version updates

  • NetBox to v4.6.10
  • Logstash to v9.5.4
  • Filebeat to v9.5.4
  • FoxIO ja4+ Zeek plugin pinned to the v1.0.1 release tag (no field-layout change from v1.0.0)
  • yq to v4.54.1
  • evtx (Windows Event Log parsing, used by Filebeat) to v0.12.3

• 🧹 Code and project maintenance

  • Refactor watch-pcap-uploads-folder.py to reduce nesting and centralize move/delete handling in file_processor(), with no change to upload behavior #1077

• 📄 Configuration changes for Malcolm (in environment variables in ./config/). The Malcolm control script (e.g., ./scripts/status, ./scripts/start) automatically handles creation and migration of variables according to ./config/env-var-actions.yml.

  • OTKB_ENRICHMENT, OTKB_URL, OTKB_TOKEN, OTKB_JSON_FIXTURE_FILE, OTKB_ENRICHMENT_TTL, OTKB_SSL_VERIFY, OTKB_ENRICHMENT_VERBOSE, OTKB_ENRICHMENT_DEBUG, and OTKB_ENRICHMENT_DEBUG_TIMINGS in logstash.env and the new logstash-secret.env (for OTKB_URL/OTKB_TOKEN) configure the pre-release MITRE OTKB enrichment integration (disabled by default)
  • SURICATA_UPDATE_SOURCES, SURICATA_UPDATE_CRON_EXPRESSION, and SURICATA_UPDATE_ENABLE_SOURCES in suricata.env control, respectively, whether the suricata-update source index is refreshed, the cron schedule for rule updates, and which indexed rule sources are enabled
    • SURICATA_UPDATE_ETOPEN has been removed and replaced by SURICATA_UPDATE_ENABLE_SOURCES, which defaults to et/open; deployments that had set SURICATA_UPDATE_ETOPEN=false should instead leave SURICATA_UPDATE_ENABLE_SOURCES blank
  • OPENSEARCH_DEFAULT_DASHBOARD in dashboards-helper.env sets the dashboard opened by default
  • NETBOX_AUTO_POPULATE_SUBNETS in netbox-common.env now accepts explicit public CIDRs (in addition to private ranges) as eligible for autopopulation

Malcolm is a powerful, easily deployable network 🖧 traffic analysis tool suite for network security monitoring 🕵🏻‍♀️.

Malcolm operates as a cluster of containers 📦, isolated sandboxes which each serve a dedicated function of the system. This makes Malcolm deployable with frameworks like Docker 🐋, Podman 🦭, and Kubernetes ⎈. Check out the Quick Start guide for examples on how to get up and running.

Alternatively, dedicated official ISO installer images 💿 for Malcolm and Hedgehog Linux 🦔 can be downloaded from Malcolm's releases page on GitHub. Due to limits on individual files in GitHub releases, these ISO files have been split 🪓 into 2GB chunks and can be reassembled with scripts provided for both Bash 🐧 (release_cleaver.sh) and PowerShell 🪟 (release_cleaver.ps1). See Downloading Malcolm - Installer ISOs for instructions.

As always, join us on the Malcolm discussions board 💬 to engage with the community, or pop some corn 🍿 and watch a video 📼.