Release Notes

7.23.0

Prelude

Release on: 2020-10-06

• Please refer to the 7.23.0 tag on
  integrations-core
  for the list of changes on the Core Checks

Upgrade Notes

• Network monitoring: enable DNS stats collection by default.

New Features

• APM: Decoding errors reported by the datadog.trace-agent.receiver.error and
  datadog.trace_agent.normalizer.traces_dropped
  contain more detailed reason tags in case of EOFs and timeouts.
• Running the agent flare with the -p flag now includes profiles for
  the trace-agent.
• APM: An SQL query obfuscation cache was added under the feature flag
  DD_APM_FEATURES=sql_cache. In most cases where SQL queries are
  repeated or prepared, this can significantly reduce CPU work.
• Secrets handles are not supported inside JSON value set through
  environment variables. For example setting a secret in a list DD_FLARE_STRIPPED_KEYS='["ENC[auth_token_name]"]'
  datadog-agent run
• Add basic support for UTF16 (BE and LE) encoding. It should be
  manually enabled in a log configuration using encoding: utf-16-be
  or encoding: utf-16-le other values are unsupported and ignored by
  the agent.

Enhancement Notes

• Add new configuration parameter to allow 'GroupExec' permission on
  the secret-backend command. Set to 'true' the new parameter
  'secret_backend_command_allow_group_exec_perm' to activate it.
• Add a map from DNS rcode to count of replies received with that
  rcode
• Enforces a size limit of 64MB to uncompressed sketch payloads
  (distribution metrics). Payloads above this size will be split into
  smaller payloads before being sent.
• APM: Span normalization speed has been increased by 15%.
• Improve the kubelet check error reporting in the output of
  agent status in the case where the agent cannot properly connect
  to the kubelet.
• Add space_id, space_name, org_id and org_name as tags to both autodiscovered
  containers as well as checks found through autodiscovery on Cloud
  Foundry/Tanzu.
• Improves compliance check status view in the security-agent status
  command.
• Include compliance benchmarks from
  github.com/DataDog/security-agent-policies in the Agent packages and
  the Cluster Agent image.
• Windows Docker image is now based on Windows Server Nano instead of
  Windows Server Core.
• Allow sending the GCP project ID under the project_id: host tag
  key, in addition to the project: host tag key, with the
  gce_send_project_id_tag config setting.
• Add kubeconfig to GCE excluded host
  tags (used on GKE)
• The cluster name can now be longer than 40 characters, however the
  combined length of the host name and cluster name must not exceed
  254 characters.
• When requesting EC2 metadata, you can use IMDSv2 by turning on a new
  configuration option (ec2_prefer_imdsv2).
• When tailing logs from container in a kubernetes environment long
  lines (>16kB usually) that got split by the container runtime
  (docker & containerd at least) are now reassembled pending they do
  not exceed the upper message length limit (256kB).
• Move the cluster-id ConfigMap creation, and Orchestrator Explorer
  controller instantiation behind the orchestrator_explorer config
  flag to avoid it failing and generating error logs.
• Add caching for sending kubernetes resources for live containers
• Agent log format improvement: logs can have kv-pairs as context to
  make it easier to get all logs for a given context Sample:
  2020-09-17 12:17:17 UTC | CORE | INFO |
  (pkg/collector/runner/runner.go:327 in work) | check:io | Done
  running check
• The CRI check now supports container exclusion based on container
  name, image and kubernetes namespace.
• Added a network_config config to the system-probe that allows the
  network module to be selectively enabled/disabled. Also added a
  corresponding DD_SYSTEM_PROBE_NETWORK_ENABLED env var. The
  network module will only be disabled if the network_config exists
  and has enabled set to false, or if the env var is set to false. To
  maintain compatibility with previous configs, the network module
  will be enabled in all other cases.
• Log a warning when a log file is rotated but has not finished
  tailing the file.
• The NTP check now uses the cloud provider's recommended NTP servers
  by default, if the Agent detects that it's running on said cloud
  provider.

Deprecation Notes

• process_config.orchestrator_additional_endpoints
  and process_config.orchestrator_dd_url are
  deprecated in favor of: orchestrator_explorer.orchestrator_additional_endpoints
  and orchestrator_explorer.orchestrator_dd_url.

Bug Fixes

• Fixed an issue where the Datadog Agent would improperly filter all
  remaining traces in a payload after a trace matching an
  ignore_resources filter was matched.
• Allow agent integration install to
  work even if the datadog agent configuration file doesn't exist.
  This is typically the case when this command is run from a
  Dockerfile in order to build a custom image from the datadog
  official one.
• Implement variable interpolation in the tagger when inferring the
  standard tags from the DD_ENV, DD_SERVICE and DD_VERSION
  environment variables
• Fix a bug that was causing not picking checks and logs for
  containers targeted by container-image-based autodiscovery. Or
  picking checks and logs for containers that were not targeted by
  container-image-based autodiscovery. This happened when several
  image names were pointing to the same image digest.
• APM: Allow digits in SQL literal identifiers (e.g. 1sad123jk)
• Fixes an issue with not always reporting ECS Fargate task_arn tag
  due to a race condition in the tag collector.
• The SUSE SysVInit service now correctly starts the Agent as the
  dd-agent user instead of root.
• APM: Allow double-colon operator in SQL obfuscator.
• UDP packets can be sent in two ways. In the "connected" way, a connect call is made first to assign the
  remote/destination address, and then packets get sent with the send function or sendto function with destination address
  set to NULL. In the "unconnected" way, packets get sent using sendto function with a non NULL destination
  address. This fix addresss a bug where network stats were not being
  generated for UDP packets sent using the "unconnected" way.
• Fix the Windows systray not appearing sometimes (bug introduced with
  6.20.0).
• The Chocolatey package now uses a fixed URL to the MSI installer.
• Fix logs tagging inconsistency for restarted containers.
• On macOS, in Agent v6, the unversioned python binaries in
  /opt/datadog-agent/embedded/bin (example: python, pip) now
  correctly point to the Python 2 binaries.
• Fix truncated cgroup name on copy with bpf_probe_read_str in OOM
  kill and TCP queue length checks.
• Use double-precision floats for metric values submitted from Python
  checks.
• On Windows, the ddtray executable now has a digital signature
• Updates the logs package to get the short image name from Kubernetes
  ContainerSpec, rather than ContainerStatus. This works around a
  known issue where the image name in the ContainerStatus may be
  incorrect.
• On Windows, the Agent now responds to control signals from the OS
  and shuts down gracefully. Coincidentally, a Windows Agent Container
  will now gracefully stop when receiving the stop command.

Other Notes

• All Agents binaries are now compiled with Go 1.14.7
• JMXFetch upgraded from
  0.38.2 to
  0.39.1
• Move the orchestrator related settings process_config.orchestrator_additional_endpoints
  and process_config.orchestrator_dd_url into
  the orchestrator_explorer section.