dmachard/DNS-collector v3.0.0-beta1

on GitHub

What's Changed

This release introduces major performance, scalability, and security improvements:

• ~40% lower memory footprint across the DNStap collector, wire-DNS decoders, and JSON serialization.
• Optional multi-worker pool (num-workers) for improved high-throughput scalability.
• New fast direct Wire DNSTap decoder
• Distroless runtime image for a smaller Docker image and reduced attack surface.
• New DNS message batching across all workers to increase throughput

        ================================================================================
                  PERFORMANCE COMPARISON: v2.5.0 vs v3.0.0
        ================================================================================
        Metric                      v2.5.0               v3.0.0              Delta
        --------------------------------------------------------------------------------
        Total Messages Processed    1000000              1000000              -
        Execution Time              1.332s               748ms                -43.82%
        Total CPU Time (User+Sys)   2.225s               712ms                -67.98%
        Peak Memory (Max RSS)       106192 KB (103.70 MB) 64784 KB (63.27 MB)  -38.99%
        Throughput                  750907.55            1336503.31           +77.99%
        ================================================================================

⚠️ Deprecations

• Worker Queue: The legacy per-worker chan-buffer-size setting is deprecated in favor of global.worker.buffer-size (default: 512) and global.worker.batch-size (default: 64). A non-blocking deprecation warning is logged if legacy options are detected.

💡 Enhancements

• feat: add UPX compression for linux, darwin and freebsd release binaries by @dmachard in #1300
• feat(docker): switch runtime image to distroless/static-debian13 with UPX compression by @dmachard in #1301

🚀 Performance

• perf: channel DNS message batching for high throughput and memory reduction by @dmachard in #1306
• feat(dnstap): zero-alloc Protobuf wire decoder for DNSTap collector by @dmachard in #1302
• perf(dnsutils,workers): binary IP representation with zero-alloc lazy formatting by @dmachard in #1303
• perf(dnsutils): fast direct JSON encoder & modular transformer encoding by @dmachard in #1296
• perf(dnsutils,workers): precompile text formatter with opcode dispatch by @dmachard in #1297
• perf(dnsutils): add zero-alloc fast-path for uncompressed DNS question decoding by @dmachard in #1298
• perf(collectors,workers): add lazy timestamp formatting by @dmachard in #1299
• perf(workers): optimize flat-json encoding across loggers and add benchmarks by @dmachard in #1294
• perf(workers): update worker telemetry channels with atomic counters by @dmachard in #1295
• perf(prom-collector): optimize prometheus metrics using atomic counters by @dmachard in #1293
• perf(dnsutils): optimize memory and CPU by using DNSMessage pointers by @dmachard in #1291
• perf(dnstap collector): add configurable worker pool for high-throughput scalability by @dmachard in #1289
• perf(dnsutils): optimize JSON serialization and DNS decoder by @dmachard in #1290
• perf(dnstap collector): add configurable socket read buffer size by @dmachard in #1288
• perf(dnstap collector): optimize processor hot path allocations and enum lookups by @dmachard in #1287

Full Changelog: v2.5.0...v3.0.0-beta1