[NOTE WELL: Please download the pmacct-1.7.7.tar.gz asset; DO NOT download any of the assets called "Source code" in either tar.gz or zip format]
- BGP, BMP, Streaming Telemetry daemons: introduced parallelization
of dump events via a configurable amount of workers where the unit
of parallelization is the exporter (BGP, BMP, telemetry exporter),
ie. in a scenario where there are 4 workers and 4 exporters each
worker is assigned one exporter data to dump. - pmtelemetryd: added support for draft-ietf-netconf-udp-notif:
a UDP-based notification mechanism to collect data from networking
devices. A shim header is proposed to facilitate the data streaming
directly from the publishing process on network processor of line
cards to receivers. The objective is a lightweight approach to
enable higher frequency and less performance impact on publisher
and receiver process compared to already established notification
mechanisms. Many thanks to Alex Huang Feng ( @ahuangfeng ) and the
whole Unyte team. - BGP, BMP, Streaming Telemetry daemons: now correctly honouring the
supplied Kafka partition key for BGP, BMP and Telemetry msg logs
and dump events. - BGP, BMP daemons: a new "rd_origin" field is added to output log/
dump to specify the source of Route Distinguisher information (ie.
flow vs BGP vs BMP). - pre_tag_map: added ability to tag new NetFlow/IPFIX and sFlow
sample_type types: "flow-ipv4", "flow-ipv6", "flow-mpls-ipv4" and
"flow-mpls-ipv6". Also added a new "is_bi_flow" true/false key to
tag (or exclude) NSEL bidirectional flows. Added as well a new
"is_multicast" true/false config key to tag (or exclude) IPv4/IPv6
multicast destinations. - maps_index: enables indexing of maps to increase lookup speeds on
large maps and/or sustained lookup rates. The feature has been
remplemented using stream-lined structures from libcdada. This is
a major work that helps preventing the unpredictable behaviours
caused by the homegrown map indexing mechanism. Many thanks to
Marc Sune ( @msune ). - maps_index: support for indexing src_net and dst_net keywords has
been added. - Added <daemon_name>_ipv6_only config directives to optionally
enable the IPV6_V6ONLY socket option. Also changed the wrong
setsockopt() IPV6_BINDV6ONLY id to IPV6_V6ONLY. - Added log function to libserdes to debug transactions with the
Schema Registry when kafka_avro_schema_registry is set. - nDPI: newer versions of the library (ie. >= 3.5) bring changes
to the API. pmacct is now aligned to compile against these. - pmacctd: added pcap_arista_trailer_offset config directive since
Arista has changed the structure of the trailer format in recent
releases of EOS. Thanks to Jeremiah Millay ( @floatingstatic )
for his patch. - More improvements carried out on the Continuous Integration
(CI) side by migrating from Travis CI to GitHub Actions. Huge
thanks to Marc Sune ( @msune ) to make all of this possible. - More improvements also carried out in the space of the Docker
images being created: optimized image size and a better layered
pipeline. Thanks to Marc Sune ( @msune ) and Daniel Caballero
( @dcaba ) to make all of this possible. - libcdada shipped with pmacct was upgraded to version 0.3.5. Many
thanks Marc Sune ( @msune ) for his work with libcdada.
! build system: several improvements carried out in this area,
ie. improved MySQL checks, introduced pcap-config tool for
libpcap, compiling on BSD/old compilers, etc. Monumental thanks
to Marc Sune ( @msune ) for his continued help.
! fix, nfacctd: improved euristics to support the case of flows
with both IPv4 and IPv6 source / destination addresses (either
or populated). Also improved euristics to distinguish event data
vs traffic data in NetFlow v9/IPFIX from Cisco 9300/9500, ASA
firewalls and Cisco 4500X.
! fix, nfacctd: improved support for initiatorOctets (IE #231) and
responderOctets (IE #232). Thanks to Esben Laursen ( @hyberdk )
for reporting the issue.
! fix, nfacctd: in NF_mpls_vpn_id_handler() double ntohl() calls
were applied for the case of 'vrfid'-encoded mpls_vpn_rd field.
! fix, sfacctd: wrong ethertype set for VLAN-tagged, MPLS-labelled
IPv6 traffic. Impacting BGP resolution among others. Thanks to
Jeremiah Millay ( @floatingstatic ) for his help resolving the
problem.
! fix, BGP, BMP daemons: parsing improvements: added a check for
BGP Open message and BGP Open Options lengths. Strengthened
parsing of Peer Up, Route Monitoring and Peer Down v4 messages.
! fix, BGP, BMP daemon: when using Avro encoding and Avro Schema
Registry, attempt to reconnect if serdes schemas are voided.
Also now checking for serdes schema definitions before doing a
serdes_schema_serialize_avro() to avoid triggering a SEGV.
Finally improved serdes logging.
! fix, BGP, Streaming Telemetry daemons: in daemon logs, summary
counters for amount of tables / entries dumped were wrong.
! fix, BGP daemon: distinguish among null and zero value AIGP
and Prefix SID attributes. Same applies for Local Preference
and MED attributes.
! fix, BMP daemon: resolved a memory leak in bgp_peers_free().
Thanks to Pether Pothier ( @pothier-peter ) for his patch. Also
resolved a leak caused by an invalid BGP message contained in a
BMP Route Message v4.
! fix, BMP daemon: correctly setting peer_ip and peer_tcp_port
JSON fields for Term messages. Also the correct bmp_router
value when bmp_daemon_parse_proxy_header feature is enabled.
! fix, BMP daemon: several encoding issues when using Apache Avro
ie. u_int64_t now correctly encoded with avro_value_set_long(),
certain u_int32_t fields switched to avro_value_set_long() due
to lack of unsignedness in Avro encoding, improved various
aspectes of Avro-JSON format output, etc.
! fix, pmtelemetryd: wrong parsing of pm_tfind() output was
leading to mistaken data attribution of UDP-based peers (always
first peer to connect was being picked).
! fix, pmtelemetryd: when set, the pidfile config directive was
not being correctly honoured.
! fix, RPKI: the RTR PDU element for maxLength is uint8, therefore
it might have been possible to transmit incorrect RTR data.
Thanks to Job Snijders ( @job ) for his patch.
! fix, SQL plugins: amended the text composition of SQL queries
that are involving latitude and longitude keys.
! fix, MySQL plugin: check for 'unix:' prefix string only when a
sql_host configuration directive is specified.
! fix, nfprobe: modernized Application Information export. Until
the previous release pmacct was adhering to aging NBAR model
whereas now NBAR2 has been implemented. Thanks to Rob Cowart
( @robcowart ) for helping out resolving this issue.
! fix, tee plugin: restored usefulness of tee_source_ip which was
broken in 1.7.6. Thanks to Jeremiah Millay ( @floatingstatic )
for reporting the issue.
! fix, maps_index: indexing of mpls_pw_id was broken. Also now,
when the feature is enabled, actual data is being referenced in
the index structure instead of creating a copy of it; thanks to
Sander van Delden ( @SanderDelden ) for reporting the memory
leak that was resulting from the copy.
! fix, kafka_common.c: solved memory leak in p_kafka_set_topic()
when Kafka session was getting in down state. Many thanks to
Peter Pothier ( @pothier-peter ) for nailing the issue.
! fix, net_aggr.[ch]: when a networks_file is specified in the
config, gracefully handle max memory structure depth; added
also de-duplication of entries.
! fix, pmacct-defines.h: if PCAP_NETMASK_UNKNOWN is not defined,
ie. in libpcap < 1.1.0, let's define it.
! fix, SO_REUSEPORT feature was being restricted to Linux only in
previous releases: now it has been unlocked to all other OS that
do support the feature.
! fix, split SO_REUSEPORT and SO_REUSEADDR setsockopt() calls.
Thanks to @eduarrrd for reporting and resolving the issue.
! fix, several code warnings catched gcc9 and clang.
- Obsoleted sql_history_since_epoch, pre_tag_map_entries and
refresh_maps configuration directives.