DataDog/datadog-agent 7.37.0

on GitHub

Prelude

Release on: 2022-06-27

• Please refer to the 7.37.0 tag on integrations-core for the list of changes on the Core Checks

Upgrade Notes

• OTLP ingest: Support for the deprecated experimental.otlp section and the DD_OTLP_GRPC_PORT and DD_OTLP_HTTP_PORT environment variables has been removed. Use the otlp_config section or the DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT and DD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT environment variables instead.
• OTLP: Deprecated settings otlp_config.metrics.report_quantiles and otlp_config.metrics.send_monotonic_counter have been removed in favor of otlp_config.metrics.summaries.mode and otlp_config.metrics.sums.cumulative_monotonic_mode respectively.

New Features

• Adds User-level service unit filtering support for Journald log collection via include_user_units and exclude_user_units.
• A wildcard (*) can be used in either exclude_units or exclude_user_units if only a particular type of Journald log is desired.
• A new troubleshooting section has been added to the Agent CLI. This section will hold helpers to understand the Agent behavior. For now, the section only has two command to print the different metadata payloads sent by the Agent (v5 and inventory).
• APM: Incoming OTLP traces are now allowed to set their own sampling priority.
• Enable NPM NAT gateway lookup by default.
• Partial support of IPv6 on EKS clusters
  • Fix the kubelet client when the IP of the host is IPv6.
  • Fix the substitution of %%host%% patterns inside the auto-discovery annotations: If the concerned pod has an IPv6 and the %%host%% pattern appears inside an URL context, then the IPv6 is surrounded by square brackets.
• OTLP ingest now supports the same settings and protocol version as the OpenTelemetry Collector OTLP receiver v0.50.0.
• The Cloud Workload Security agent can now monitor and evaluate rules on bind syscall.
• [corechecks/snmp] add scale factor option to metric configurations
• Evaluate memory.usage metrics based on collected metrics.

Enhancement Notes

• APM: DD_APM_FILTER_TAGS_REQUIRE and DD_APM_FILTER_TAGS_REJECT can now be a literal JSON array. e.g. ["someKey:someValue"] This allows for matching tag values with the space character in them.
• SNMP Traps are now sent to a dedicated intake via the epforwarder.
• Update SNMP traps database to include integer enumerations.
• The Agent now supports a single com.datadoghq.ad.checks label in Docker, containerd, and Podman containers. It merges the contents of the existing check_names, init_configs (now optional), and instances annotations into a single JSON value.
• Add a new Agent telemetry metric autodiscovery_poll_duration (histogram) to monitor configuration poll duration in Autodiscovery.
• APM: Added /config/set endpoint in trace-agent to change configuration settings during runtime. Supports changing log level(log_level).
• APM: When the X-Datadog-Trace-Count contains an invalid value, an error will be issued.
• Upgrade to Docker client 20.10, reducing the duration of docker check on Windows (requires Docker >= 20.10 on the host).
• The Agent maintains scheduled cluster and endpoint checks when the Cluster Agent is unavailable.
• The Cluster Agent followers now forward queries to the Cluster Agent leaders themselves. This allows a reduction in the overall number of connections to the Cluster Agent and better spreads the load between leader and forwarders.
• The kube_namespace tag is now included in all metrics, events, and service checks generated by the Helm check.
• Include install_info to version-history.json
• Allow nightly builds install on non-prod repos
• Add a kubernetes_node_annotations_as_tags parameter to use Kubernetes node annotations as host tags.
• Add more detailed logging around leadership status failures.
• Move the experimental SNMP Traps Listener configuration under network_devices.
• Add support for the DNS Monitoring feature of NPM to Linux kernels older than 4.1.
• Adds segment_name and segment_id tags to PCF containers that belong to an isolation segment.
• Make logs agent additional_endpoints reliable by default. This can be disabled by setting is_reliable: false on the additional endpoint.
• On Windows, if a datadog.yaml file is found during an installation or upgrade, the dialogs collecting the API Key and Site are skipped.
• Resolve SNMP trap variables with integer enumerations to their string representation.
• [corechecks/snmp] Add profile static_tags config
• Report telemetry metrics about the retry queue capacity: datadog.agent.retry_queue_duration.capacity_secs, datadog.agent.retry_queue_duration.bytes_per_sec and datadog.agent.retry_queue_duration.capacity_bytes
• Updated cloud providers to add the Instance ID as a host alias for EC2 instances, matching what other cloud providers do. This should help with correctly identifying hosts where the customer has changed the hostname to be different from the Instance ID.
• NTP check: Include /etc/ntpd.conf and /etc/openntpd/ntpd.conf for use_local_defined_servers.
• Kubernetes pod with short-lived containers do not have log lines duplicated with both container tags (the stopped one and the running one) when logs are collected. This feature is enabled by default, set logs_config.validate_pod_container_id to false to disable it.

Security Notes

• The Agent is built with Go 1.17.11.

Bug Fixes

• Updates defaults for the port and binding host of the experimental traps listener.
• APM: The Agent is now performing rare span detection on all spans, as opposed to only dropped spans. This change will slightly reduce the number of rare spans kept unnecessarily.
• APM OTLP: This change ensures that the ingest now standardizes certain attribute keys to their correct Datadog tag counter parts, such as: container tags, "operation.name", "service.name", etc.
• APM: Fix a bug where the APM section of the GUI would not show up in older Internet Explorer versions on Windows.
• Support dynamic Auth Tokens in Kubernetes v1.22+ (Bound Service Account Token Volume).
• The %%host%% autodiscovery tag now works properly when using containerd, but only on Linux and when using IP v4 addresses.
• Enhanced the coverage of pause-containers filtering on Containerd.
• APM: Fix the loss of trace metric container information when large payloads need to be split.
• Fix cri check producing no metrics when running on OpenShift / cri-o.
• Fix missing health status from Docker containers in Live Container View.
• Fix Agent startup failure when running as a non-privileged user (for instance, when running on OpenShift with restricted SCC).
• Fix missing container metrics (container, containerd checks and live container view) on AWS Bottlerocket.
• APM: Fixed an issue where "CPU threshold exceeded" logs would show the wrong user CPU usage by a factor of 100.
• Ensures that when kubernetes_namespace_labels_as_tags is set, the namespace labels are always attached to metrics and logs, even when the pod is not ready yet.
• Add missing support for UDPv6 receive path to NPM.
• The agent workload-list --verbose command and the workload-list.log file in the flare do not show containers' environment variables anymore. Except for DD_SERVICE, DD_ENV and DD_VERSION.
• Fixed a potential deadlock in the Python check runner during agent shutdown.
• Fixes issue where trace-agent would not report any version info.
• The DCA and the cluster runners no longer write warning logs to /tmp.
• Fixes an issue where the Agent would panic when trying to inspect Docker containers while the Docker daemon was unavailable or taking too long to respond.

Other Notes

• Exclude teradata on Mac agents.