Estimated end-of-life date, accurate to within three months: 07-2027
See the support level definitions for more information.
Upgrade Notes
- tracing
- The
Hooksclass (config.<integration>.hooks) is deprecated and will be removed in v5.0. All hook methods (register(),on(),deregister(),emit()) are now no-op and no longer affect span behavior. To interact with spans, useddtrace.trace_utils.get_current_span()orddtrace.trace_utils.get_current_root_span()instead.
- The
New Features
- LLM Observability
- Adds tracing of
initializerequests and their responses on modelcontextprotocol/python-sdk servers. - This introduces automatic
source:oteltagging for evaluations when OpenTelemetry (OTel) tracing is enabled whenDD_TRACE_OTEL_ENABLED=trueis set. This tag allows the backend to wait for OTel span conversion before processing evaluations. - Changes format of MCP server tool call span input, output, and tags to include the full request and response objects.
- Reasoning token counts are now captured from VertexAI responses.
- Adds tracing of
- profiling
- Add support for
asyncio.BoundedSemaphorelock type profiling in Python Lock Profiler. - Add support for
asyncio.Conditionlocking type profiling in Python. The Lock profiler now provides visibility intoasyncio.Conditionusage, helping identify contention in async applications using condition variables. - Add support for
asyncio.Semaphorelock type profiling in Python Lock Profiler.
- Add support for
- AAP
- Add business logic event detection for Stripe. This feature instruments the payment intent and checkout session creation actions as well as the payment_intent.succeeded, payment_intent.payment_failed and payment_intent.canceled events.
- Proxy inferred spans now contain events when AppSec is enabled so that they are reported on the App and API Protection Endpoint Catalog.
- DBM
- Adds container tags support
- DSM
- Adds container tags support
- tracing
- Proxy inferred spans now differentiate between API Gateway v1 and v2 apis by emitting the
aws.httpapispan name for v2 apis when the API Gateway sets thex-dd-proxyheader toaws-httpapi. Additionally, the taghttp.routeand the resource name of the span now contains the api resource path instead of the path when propagated with thex-dd-proxy-resource-pathheader.
- Proxy inferred spans now differentiate between API Gateway v1 and v2 apis by emitting the
- vllm
- Introduces tracing and LLM Observability support for vLLM V1 engine. Requires vLLM >= 0.10.2. See the docs for more information.
Bug Fixes
- CI Visibility
- This fix resolves an issue where code coverage instrumentation in Python 3.9 would raise an exception while handling line numbers in some corner cases.
- crash tracking
- Passing in all environment variables to the crashtracker receiver process caused conflicts with export location derivation. This change only passes in the DD_CRASHTRACKING_ERRORS_INTAKE_ENABLED environment variable.
- falcon
- Fixes
DD_TRACE_REMOVE_INTEGRATION_SERVICE_NAMES_ENABLEDsupport, which was previously ignored.
- Fixes
- tracing
- Proxy inferred spans used to omit the
https://scheme prefix as part of thehttp.urltag; this caused the entire url to be parsed as the http path. - psycopg
- Fixes an OperationalError that occurred when patching a closed psycopg3 connection.
- Proxy inferred spans used to omit the
- dynamic instrumentation
- fixed an issue that caused condition expressions containing
isDefinedto result in an evaluation error. - fixed an issue that prevented autocomplete information to be extracted from the running application.
- fixed an issue that caused condition expressions containing
- openfeature
- This fix resolves an issue where Feature Flagging and Experimentation (FFE) was not receiving remote configuration in forking web server environments (gunicorn, uWSGI). This caused the OpenFeature provider to return default values instead of configured feature flags. FFE is now properly registered as a product during tracer initialization (when
DD_EXPERIMENTAL_FLAGGING_PROVIDER_ENABLED=true), ensuring remote configuration is received before process forking occurs. - Fix exposure event deduplication to use (flag_key, subject_id) as cache key instead of (flag_key, variant_key, allocation_key). This ensures different users each receive their own exposure event while still deduplicating repeated evaluations for the same user. Also adds LRU eviction to prevent unbounded memory growth and respects the do_log flag from flag metadata.
- This fix resolves an issue where Feature Flagging and Experimentation (FFE) was not receiving remote configuration in forking web server environments (gunicorn, uWSGI). This caused the OpenFeature provider to return default values instead of configured feature flags. FFE is now properly registered as a product during tracer initialization (when
- ray
- This fix resolves an issue where
Nonemetadata in Ray job submission caused a crash.
- This fix resolves an issue where
- openai_agents
- This fix resolves an issue where a missing active span caused an
AttributeErrorwhen callingtag_agent_manifest.
- This fix resolves an issue where a missing active span caused an
- profiling
- This fix resolves an issue where the Lock profiler would not call the necessary initialization function, which would sometimes result in crashes.
- the Profiler now always uses the name of leaf tasks for the "Task name" label. Previously, one of the Stacks would be labelled with the parent task's name, which would lead to inconsistent behaviour across runs.
- Fixes a bug where code that sub-classes our wrapped locks crashes with
TypeErrorduring profiling. One example of this is neo4j'sAsyncRLock, which inherits fromasyncio.Lock: https://github.com/neo4j/neo4j-python-driver/blob/6.x/src/neo4j/_async_compat/concurrency.py#L45 - a bug causing crashes when using
uvloopand forking has been resolved. - This improves the accuracy of stacks for on-CPU asyncio Tasks by reducing the odds of Frames for a Task polluting the stack of other Tasks.
- the build now uses the correct location for the native extension module. Previously, linking would work correctly in tests, b