JA4+ Zeek scripts updated to Zeek Plugin

C++ Port by @mmguero in #313
Below are the pull request comments from @mmguero

Summary

Replaces the pure Zeek-script JA4+ implementation with a compiled Zeek plugin. JA4, JA4S, JA4H, JA4SSH, and JA4T (raw packet parsing) are now C++ BiF functions; JA4L and JA4D stay pure script since neither does anything performance-critical enough to justify the port.

Repo restructured under zeek/ to separate the plugin source from repo-level docs/licensing.

Requirements change

Zeek 5.0+ → Zeek 7.0+. Needed for the C++20 Broker headers the plugin depends on. Also requires a C++20 compiler (Clang 19+ or GCC 12+) and CMake 3.15+.

Feature parity

Diffed every script against the current upstream pure-script package field by field, then verified behavior against 81 real pcaps across all 7 modules.

For now, ja4x matches upstream's current state exactly: still an empty placeholder gated behind a disabled-by-default option, pending future implementation. Not implemented here either; this fork isn't ahead or behind upstream on JA4X.

Build/packaging

The old package was pure script installed via zkg's script mechanism. This is a compiled plugin, which is a different install path end to end, and most of the real work here was getting that path to actually function:

• zkg.meta needed zeek/ prefixes on script_dir, test_command, build_command after the restructure, plus a plugin_dir field (without it zkg builds the BiFs but never registers the plugin — invisible to zeek -N).
• Missing plain README (no extension) required by Zeek's ConfigurePackaging.cmake.
• Fixed a module-scoping issue: the entry-point loader declared module FINGERPRINT; before loading external base Zeek modules (base/protocols/conn, ssl, http/main, quic). Since Zeek's @load doesn't reset module context, unnamespaced base-script functions were getting declared inside FINGERPRINT:: instead of globally, breaking parts of Zeek's own base scripts the moment those modules got touched downstream. Fixed by front-loading all external @loads ahead of the module declaration.
• Fixed a duplicate-script-install issue: script_dir in zkg.meta was installing scripts a second time via zkg's site-package mechanism on top of the plugin's own auto-load bundle, causing double-declaration errors under local/site policy. Resolved by pointing script_dir at an empty stub inside zeek/.
• Test suite had path drift from the restructure (TestDirs, baseline directory names, a stale plugin identifier in the show-plugin test).

Runtime enable/disable

Individual JA4+ methods (JA4_enabled, JA4S_enabled, JA4H_enabled, JA4SSH_enabled, JA4T_enabled, JA4TS_enabled, JA4L_enabled, JA4D_enabled) can be toggled at runtime without rebuilding, via -e or a redef script:

zeek -e 'redef FINGERPRINT::JA4H_enabled = F;' -C -r trace.pcap

Disabling a method stops it from computing and populating its field(s), but the log column(s) it uses still exist — they'll simply be empty.

The _raw flags (JA4_raw, JA4S_raw, JA4H_raw) work differently: they control whether extra (non-hashed) log columns exist at all. These must be set in config.zeek before building — they cannot be toggled via runtime redef.

Test suite

Consolidated coverage from both the upstream script-only suite and this fork's own suite. Added ja4-conn-ipv6, ja4-conn-quic, ja4-conn-tls3 for transport diversity upstream had that this fork didn't. Added seven *-disabled tests, one per _enabled flag, verifying the runtime-toggle behavior end to end. Added ja4-dtls to lock in correct ja4/ja4s proto-character behavior on DTLS traffic — none of the existing pcaps in the suite exercised this path, so the fix had no regression coverage until now.

Skipped porting upstream's full-log-diff style tests where they'd duplicate an existing fork test on the same pcap/module — kept this fork's narrower zeek-cut-on-specific-fields approach, since a full-log diff baseline breaks on any unrelated Zeek default-field change.

19 tests total, all passing.

Validation

Cross-validated this fork against current upstream across 81 real pcaps, all 7 fingerprint modules each (~570 comparisons). Zero non-zero exit codes, zero unexpected stderr on either side. Field-level output is byte-identical to upstream across the entire corpus. Confirmed matching record counts across all log types to rule out silently dropped connections producing a false match from both sides being empty.

Tested on both amd64 and arm64, against Zeek 7.0.x, 8.0.x, and 8.2.x.

Performance

This was the actual original goal of the C++ port. parse_tcp_options (raw byte-level TCP option parsing in ja4t.cc) is a legitimate win: Zeek script bytestring handling is slow for this kind of work and it runs on every SYN/SYN-ACK. The other four ported modules (JA4/JA4S/JA4H/JA4SSH) are mostly string assembly around sha256_hash, which is already a native BiF even in the pure-script version. For these, C++ port speeds up the orchestration around the hash, not the hash itself, so the gains there are real but probably more modest than "rewrite in C++" implies.

Performance testing across 3,181 directly comparable PCAP run (pulled from various data sets all over the internet) showed an overall reduction in processing time from 6,965 seconds to 5,840 seconds, or approximately 16.2%. The improvement was most pronounced on longer-running captures: runs that previously took 20–59 seconds completed about 40.4% faster on average, while runs taking at least one minute improved by about 33.8%. Shorter 2–19 second runs improved more modestly, by about 8.4%. Sub-second and one-second runs were effectively unchanged within the precision of the recorded timing data.

Some of the overall improvement was driven by a few substantial outliers. The largest capture dropped from 862 seconds to 277 seconds, saving 585 seconds, while another fell from 188 seconds to 49 seconds. Even with the largest outlier excluded, however, the new implementation remained approximately 8.8% faster overall. Notably, every run that previously took at least 20 seconds completed faster in the new results, with no exit-code changes across the dataset.

Overall, performance appears to be modestly improved across the broader dataset, although the degree of improvement is necessarily traffic-specific. Captures whose traffic exercises the affected fingerprints see the greatest gains. For a more fine-grained analysis of where execution time is being spent, it may be useful to repeat targeted runs under perf and generate flame graphs using zeek-perf-support, which provides Zeek profiling support analogous to Python’s PYTHONPERFSUPPORT.

A separate round of testing with the Zeek Abstract Machine (ZAM) enabled would also be interesting. ZAM is Zeek’s optional script-optimization engine; rather than recursively interpreting Zeek script syntax trees directly, it compiles them into a lower-level representation intended to execute more efficiently. It can be enabled with -O ZAM, although the benefit depends on how much of a workload is spent executing Zeek scripts rather than the core event engine or complex built-in functions. Zeek’s documentation notes that gains are often around 10–15%, but ZAM also incurs several seconds of code-generation overhead at startup. Any comparison—particularly for short PCAP runs—should therefore distinguish startup and compilation time from the actual traffic-processing interval, such as by measuring between the zeek_init and zeek_done events.

Provenance

This C++ port of the JA4+ Zeek plugins was developed by Battelle Energy Alliance (BEA) / Idaho National Laboratory (INL), in coordination with SEI, and is contributed to FoxIO under the terms agreed between BEA/INL/SEI and FoxIO. Upon acceptance, these plugins are made available under the FoxIO License 1.1, consistent with the existing JA4+ Zeek scripts in this repository, and are subject to that license the same as any other code here. This contribution does not grant BEA/INL/SEI any ownership interest in FoxIO's JA4+ technology or repository, and does not obligate FoxIO to share commercial licensing revenue with BEA/INL/SEI or any other contributor. BEA/INL will make a good faith effort to support future contributions, including addressing bugs, compatibility issues, and other issues, subject to available resources.