github DataDog/dd-trace-py v4.14.0rc1
4.14.0rc1

pre-release3 hours ago

Release Notes

New Features

  • CI Visibility: Adds a new ci.pipeline.display_name tag for the Buildkite provider, populated from the BUILDKITE_PIPELINE_NAME environment variable.
  • Data Streams Monitoring: Adds an optional tags argument to ddtrace.data_streams.set_produce_checkpoint and ddtrace.data_streams.set_consume_checkpoint. Pass a list of "key:value" strings to attach additional edge tags to a manual checkpoint alongside the automatically set type, topic, and direction tags—for example, set_produce_checkpoint("eventbridge", detail_type, carrier_set, tags=["exchange:my-bus"]). Existing calls that do not pass tags are unaffected.
  • LLM Observability: Adds LLMObs.list_experiments() to query experiments by logical name and metadata containment (for example, {"tags": ["git.commit.sha:abc123"]}). It returns run status and aggregate data (evaluations, token costs, and error rates), allowing CI/CD workflows to compare a run against a baseline commit and gate on the result.
  • LLM Observability: Adds support for manually versioning agents. Pass version when starting an agent span, either with LLMObs.agent(name="my-agent", version="v3") or with the @agent(version="v3") decorator. For agents created by an auto-instrumented framework, use LLMObs.annotation_context(agent={"version": "v3"}). This sets the agent_version tag on agent spans. The agent argument accepts either a plain dictionary or the new ddtrace.llmobs.Agent TypedDict class.
  • LLM Observability: Adds LLMObs.submit_feedback() for submitting end-user feedback associated with a span, trace, session, or customer-defined feedback join key.
  • openfeature: Adds agentless delivery for the Feature Flagging and Experimentation (FFE) OpenFeature provider, loading Universal Flag Configuration directly from Datadog over HTTPS without requiring a Datadog Agent. The agentless source is now the default and requires DD_API_KEY; set DD_FEATURE_FLAGS_CONFIGURATION_SOURCE=remote_config to continue loading configuration through the Datadog Agent's Remote Configuration.
  • opentelemetry: OTLP trace export now supports the http/protobuf protocol (the default) and http/json, selected via the OTEL_EXPORTER_OTLP_TRACES_PROTOCOL environment variable.
  • tracing: Adds Span.remove_tag(key) and Span.remove_metric(key) for removing a previously set tag or metric from a span. Both are no-ops if the key is not set.

Bug Fixes

  • AAP: Fixes incorrect request fingerprints when a web application is wrapped by same-service tracing middleware.
  • botocore: Fixes an issue where LLM Observability spans for Bedrock invoke_model_with_response_stream calls reported all token usage metrics as 0. Token counts are now extracted from the streamed response and set on the span.
  • CI Visibility: Fixes an issue where code coverage collected for Intelligent Test Runner (ITR) included files from site-packages or dist-packages directories.
  • CI Visibility: Fixes an issue where the session status was reported as pass instead of fail when pytest exited with a non-OK, non-NO_TESTS_COLLECTED exit code (for example, because of collection errors, interruptions, or internal errors).
  • Code Security: Fixes an issue where applications could crash when asynchronous tasks accessed request data after the originating request completed.
  • django: Fixes an issue (#19454) where Django processes that never serve HTTP requests, such as Celery and Dramatiq workers, used substantially more memory and took longer to start. API endpoint discovery now runs when the WSGI or ASGI application is built rather than during django.setup().
  • gevent: Fixes an issue where running an application with ddtrace-run (or single-step instrumentation) could raise a PydanticSchemaGenerationError at import time for models that use forward references when gevent is installed and the application runs on CPython 3.14 or later. The annotationlib module, which now owns the ForwardRef class on CPython 3.14, is preserved during module cloning so that pydantic resolves forward references correctly.
  • kombu: Fixes an issue where messages published to the RabbitMQ default (unnamed) exchange were not linked to their consumers in the Data Streams Monitoring map.
  • OpenTelemetry logs: Fixes an issue where enabling OpenTelemetry logs collection (DD_LOGS_OTEL_ENABLED=true) could cause the tracer to export its own internal log records, producing a growing volume of exported logs. The tracer's own log records and the OpenTelemetry exporter's log records are now excluded from OpenTelemetry logs export, while application logs continue to be collected and exported as before.
  • profiling: Fixes an issue where applications using uWSGI with --lazy-apps and --skip-atexit could crash while workers shut down on Python 3.12 or later.
  • profiling: Fixes an issue where child spans created from propagated tracing contexts could lose local-root and endpoint attribution when greenlets shared a thread.
  • pytorch: Fixes an issue where the pytorch.rank span could end too soon.
  • requests: Fixes an issue where distributed tracing context propagated on outbound requests calls pointed to the wrong span when the urllib3 integration was also enabled (for example, via DD_TRACE_URLLIB3_ENABLED=true), causing those requests to be missing or to show incorrect latency in downstream traces.
  • tracing: Fixes an issue where the sampling decision of an incoming distributed trace was discarded, reported as a propagation error on the local root span, and logged as a warning when the trace was sampled by a mechanism unknown to this version of the library. Well-formed sampling decisions are now propagated unchanged.
  • tracing: Fixes an issue where OpenTelemetry thread-context record publication could not be disabled on Linux. Publication can now be disabled by setting the experimental DD_TRACE_OTEL_CTX_ENABLED environment variable to false.
  • tracing: Fixes an issue where OTLP trace export failures reported the Datadog Agent intake URL instead of the configured OTLP endpoint.
  • tracing: Fixes an issue where _dd.svc_src was incorrectly set to either "m" or the service name.

Other Changes

  • botocore: Payload-tagging redaction now uses python-jsonpath in place of the vendored jsonpath-ng and ply, so ddtrace no longer ships ply, which is unmaintained and flagged by dependency scanners for CVE-2025-56005. Existing redaction paths continue to work.

Don't miss a new dd-trace-py release

NewReleases is sending notifications on new releases.