DataDog/dd-trace-py v4.17.0rc1

4.17.0rc1

on GitHub

Estimated end-of-life date, accurate to within three months: 05-2027
See the support level definitions for more information.

Upgrade Notes

• CI Visibility
  • Removes the legacy pytest plugin. The DD_PYTEST_USE_NEW_PLUGIN and DD_PYTEST_USE_NEW_PLUGIN_BETA environment variables no longer have any effect; the pytest plugin is always used. Remove these environment variables from your configuration to silence the deprecation warning emitted while they are set.
• dd-trace-py 4.17 adds support for Python 3.15. Most of the library is tested and ready on 3.15. Some products and integrations are not yet validated on Python 3.15. You can track progress in #17809.

New Features

• AAP
  • Adds Server-Side Request Forgery (SSRF) exploit prevention and downstream request analysis for outgoing requests made with the aiohttp client.
  • Adds threat detection and request blocking for aiohttp web applications traced with ddtrace.contrib.aiohttp.trace_app. Request bodies sent with chunked transfer encoding or compression, or larger than the application's client_max_size, are not inspected.
  • Adds SQL injection protection for byte queries, including bytearray and memoryview queries, in SQL integrations and Django MySQL backends. Queries are inspected using the connection's encoding, with UTF-8 inspection for PostgreSQL SQL_ASCII connections.
• AI Guard
  • Adds an optional tags argument to AIGuardClient.evaluate(). Each key/value pair in the dictionary is set as a tag on the AI Guard span, so evaluations can be filtered by custom fields in queries, e.g. client.evaluate(messages, tags={"tenant": "acme"}). Keys starting with ai_guard. are reserved and ignored.
  • Introduces filters and callbacks through middlewares in ddtrace.aiguard.middleware, to control which requests AI Guard evaluates and to run custom logic around each evaluation. See the AI Guard documentation for how to use them.
• algoliasearch
  • This introduces tracing support for algoliasearch 3.x.
• elevenlabs
  • Adds LLM Observability support for Python Agents conversations over WebSocket, including conversation audio, transcripts, and client tools. Set DD_TRACE_ELEVENLABS_ENABLED=false to disable automatic instrumentation. For improved user-speaking colors, enable vad_score in the agent's Advanced settings under Client Events. Audio playback remains available without this optional event.
• pymongo
  • Adds support for PyMongo v4.18
  • Adds Python 3.15 test coverage for current PyMongo releases.
• LLM Observability
  • The agent version set through LLMObs.agent(version=...), @agent(version=...), LLMObs.annotate(agent={"version": ...}) or LLMObs.annotation_context(agent={"version": ...}) is now also reported as an agent_version tag on child spans of that agent within the same process.
  • The root LLMObs span now includes http.client_ip and network.client.ip tags when AppSec or DD_TRACE_CLIENT_IP_ENABLED is active and the web framework instruments the request. No additional LLMObs configuration is required.
  • add Amazon Nova 2 Sonic support for the asynchronous AWS SDK for Bedrock Runtime. Conversations include per-turn traces, transcripts, text and audio token usage, tool calls, and playable audio, with speech phases for conversation playback and response-latency measurement.
  • The OpenAI integration now reports ephemeral_30m_input_tokens and ephemeral_12h_input_tokens metrics for Responses API calls.
  • Adds support for tracing OpenAI Decisions API calls (client.decisions.create), capturing the input, questions, answers, and token usage.
  • Adds LLMObs.routing_context(), a context manager that sends LLM Observability spans and evaluation metrics created within it to other Datadog organizations instead of the configured one. Pass dd_api_key (and optionally dd_site) to route to one organization, or targets to send the same data to several. Routed data is sent directly to the LLM Observability intake; APM spans are not routed.
  • The OpenAI and Anthropic integrations now track server-side tool usage on LLM spans. This includes web_search_count for OpenAI Responses API and Anthropic web searches, and storage_search_count for OpenAI file searches.
  • Adds support for submitting LLM Observability spans for TypeSafe system_one calls, capturing the request and response bodies and token usage. Images are recorded by reference, not by content.
• AWS Bedrock Runtime
  • Adds reported input and output text token counts to Nova Sonic 2 spans in LLM Observability alongside audio token counts.
• OpenAI
  • Adds reported audio, text, image, and cached token counts to Realtime spans in LLM Observability for detailed token breakdowns.
• mcp
  • LLM Observability MCP server spans, including tool call spans, now include the calling client's self-reported name, title, and version as the mcp_client_name, mcp_client_title, and mcp_client_version tags. Tool call spans are not tagged for servers on mcp<2 using stateless streamable HTTP, where the client's identity is unavailable.
  • Adds support for mcp 2.x, including clients and servers using the stateless 2026-07-28 MCP protocol. Previously, no APM or Agent Observability spans were created for MCP clients or servers when mcp 2.x was installed.
• tracing
  • Adds DD_TRACE_HTTP_CLIENT_ERROR_STATUSES to configure which HTTP client response status codes mark a span as an error when DD_TRACE_OTEL_SEMANTICS_ENABLED=true. When it is not set, client status codes of 400 and above are errors.
  • Adds support for OpenTelemetry HTTP semantic conventions on HTTP client spans, enabled with DD_TRACE_OTEL_SEMANTICS_ENABLED=true. When enabled, client spans use OpenTelemetry attribute names and low-cardinality resource names; the default remains the existing Datadog behavior.
  • Adds support for OpenTelemetry HTTP semantic conventions on HTTP server spans when DD_TRACE_OTEL_SEMANTICS_ENABLED=true.
  • Adds shared OpenTelemetry HTTP span behavior for upcoming client and server integrations, enabled with DD_TRACE_OTEL_SEMANTICS_ENABLED. See the OpenTelemetry API docs for details.
  • Adds gRPC support for OTLP trace export. Set OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=grpc or OTEL_EXPORTER_OTLP_PROTOCOL=grpc to use gRPC; http/protobuf remains the default for traces. Applications that use the global gRPC setting but require HTTP trace export must set OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf. gRPC trace export currently supports plaintext http:// endpoints only.
• profiling
  • Adds experimental support for installing Profiler's hooks at startup without starting the profiler, so that a profiler can be started later at runtime. This option is disabled by default, is not yet recommended for production use, and its behavior may change in future releases.
• CI visibility
  • Reports whether Test Impact Analysis skipped tests in a suite so affected durations can be excluded from comparisons. This is available for serial runs of the current pytest integration, without requiring code coverage collection.
  • Adds pytest.mark.dd_prevent_failure_suppression to override Datadog quarantine and disabling, and pytest.mark.dd_no_auto_retry to exclude Datadog automatic retries. Both markers support test functions, classes, modules, and individual parameter cases while preserving explicit pytest plugin retries.
• typesafe
  • Adds tracing support for TypeSafeClient.system_one and AsyncTypeSafeClient.system_one calls in the TypeSafe Python SDK (typesafe-sdk).

Security Issues

• internal
  • HTTPS connections made by ddtrace's native components now reject certificate chains that pass through a small set of intermediate certificate authorities distrusted due to security concerns.
• AAP
  • Fixes an issue where building ddtrace from source with Python optimizations enabled (-O or PYTHONOPTIMIZE) skipped the integrity check of the downloaded libddwaf library. The library is now checked against checksums pinned in ddtrace rather than ones downloaded alongside it.

Bug Fixes

• AI Guard
  • Fixes an issue where setting DD_AI_GUARD_TIMEOUT below 1000 resulted in a request timeout of 0, because the value is applied in whole seconds. Such values now log a warning at startup and fall back to the default of 10000.
  • Fixes an issue where streamed LangChain chat model and LLM responses were not evaluated when DD_AI_GUARD_ANALYZE_STREAM_RESPONSES_ENABLED is true. The stream is now read and evaluated before any chunk is returned, and a block returns no chunk.
  • Fixes an issue where, with DD_AI_GUARD_ANALYZE_STREAM_RESPONSES_ENABLED set to true, a LangChain model that streams delivered tokens to callbacks (for example astream_events() or LangGraph stream_mode="messages") before AI Guard evaluated the response. Callbacks now receive the tokens after the evaluation, and none when it blocks.
  • Fixes an issue where the Strands Agents integration raised a ValueError when an agent invocation finished in a different asyncio task, and where OpenAI and Anthropic calls in the task that started the invocation were then no longer evaluated.
  • Fixes an issue where streaming OpenAI Responses API calls (responses.create(stream=True)) were not evaluated when DD_AI_GUARD_ANALYZE_STREAM_RESPONSES_ENABLED was unset or false. Request inputs are now evaluated, matching Chat Completions and Anthropic streaming calls, so these calls can now be blocked when blocking is enabled. To keep evaluating without blocking, set DD_AI_GUARD_BLOCK=false; to disable AI Guard for OpenAI, set DD_AI_GUARD_OPENAI_ENABLED=false.
• aws_lambda
  • Rotates the runtime ID and refreshes registered components when an AWS Lambda MicroVM application starts after restore.
• CI Visibility
  • Fixes excessive CPU usage that could slow down or time out test jobs when Code Coverage is enabled with pytest-xdist.
  • Fixes an issue where pytest test discovery reports success despite collection errors. With DD_TEST_OPTIMIZATION_DISCOVERY_ENABLED enabled, collection errors now return exit code 2 while preserving partial discovery results, unless --continue-on-collection-errors is supplied.
  • Fixes excessive memory use when generating Code Coverage reports for large projects when using coverage.py, particularly when tests run with multiple workers.
  • Fixes an issue where pytest runs using copied or restored execution contexts (e.g. Django async tests via asgiref's async_to_sync) could end with an internal coverage error, or silently lose or misattribute the coverage data of the tests that use them.
  • Fixes an issue where long test error stack traces and messages were cut off at the end, hiding the failing frame and exception message. When error.stack or error.message exceed 5000 characters, the last 5000 characters are now kept.
  • Fixes pytest sessions being reported as passed instead of skipped when no tests are collected.
• profiling
  • Optimizes unwinding async tasks by removing duplicate intermediate data structure creation
  • Fixes an issue where starting a thread blocks forever when the profiler is enabled and the thread name contains a lone surrogate or a null character.
  • Fixes a crash under uWSGI on Python 3.12+ when the application is loaded in the master process without the lazy-apps, py-call-uwsgi-fork-hooks or py-call-osafterfork option. Native frame attribution and exception profiling are now disabled in these workers. Enable lazy-apps or py-call-uwsgi-fork-hooks to keep them.
  • fixes a crash where mem profiler state was not reset during shutdown, so allocations during shutdown could cause a crash
  • A crash that could happen at exit when using memory profiling has been fixed.
  • Fixes an issue where exception profiling failed to start when another tool had already claimed the sys.monitoring slot used by the profiler, by sharing a single monitoring tool with error tracking and falling back to another free slot when one is taken.
  • Profilers failing to start now clean up after themselves instead of keeping half-initialised resources around.
  • On Python 3.14 and later, prevents stack samples from being attributed to stale Python frames after code objects are replaced.
  • a bug that could raise RecursionError when pickling a profiler object has been fixed.
• botocore
  • Fixes an issue where Data Streams Monitoring did not track SQS messages consumed with a JSON body that is not an object, such as an array, string, or number.
  • Fixes an issue where Data Streams Monitoring under-reported end-to-end latency for pathways that pass through a Kinesis consumer.
• error tracking
  • Fixes an issue where handled exceptions from modules listed in DD_ERROR_TRACKING_HANDLED_ERRORS_INCLUDE were not reported on Python 3.12 and later when the first handled exception occurred while the module was still being imported, causing all later handled exceptions from the same module to also be missed.
  • Fixes an issue where an internal handled error reporting failure can interrupt application exception handling on Python 3.12 and later.
• AAP
  • Fixes an issue where services were reported to Datadog as having App and API Protection enabled when it was not active, either because DD_APPSEC_ENABLED was unset and AAP had not been activated through Remote Configuration, or because AAP failed to load at startup.
• tracing
  • Fixes an issue where traces that do not match any DD_TRACE_SAMPLING_RULES rule and have no service-specific Agent rate are sampled at 100% instead of using the default rate provided by the Datadog Agent.
  • Fixes an issue where per-integration HTTP settings, such as config.<integration>.http.trace_headers(), were ignored when set before the integration was enabled.
  • Fixes an issue where AWS Lambda invocations without an HTTP URL are assigned an HTTP endpoint when trace resource renaming is enabled.
  • Fixes an issue where baggage in OTEL_PROPAGATORS is reported as unsupported and ignored.
  • Fixes an issue where OTLP log and metric resources use the deprecated deployment.environment attribute instead of deployment.environment.name for DD_ENV.
  • Fixes a rare ZeroDivisionError raised from span.finish() when multiple threads write traces concurrently.
  • Fixes an issue where the dd= list-member of the W3C tracestate header could exceed 256 characters, causing proxies that enforce the W3C limit to drop the whole tracestate header.
  • Fixes a memory leak where a contextvars copy made inside a @tracer.wrap function kept the local variables of the finished call in memory.
  • Reduces the overhead of setting ordinary string-valued span tags.
  • Fixes an issue where service, environment, and version values configured through OTEL_RESOURCE_ATTRIBUTES can differ between traces, logs, and metrics. These unified service tags are now applied consistently across all three signals.
• aiohttp
  • Fixes an issue where tracing HTTP server requests produces repeated callback errors when keep-alive connections close.
  • Fixes an issue where every request to an application traced with trace_app failed with KeyError: 'disable_stream_timing_for_mem_leak' when the aiohttp integration was not patched, for example with DD_TRACE_ENABLED=false.
  • Fixes an issue where the http.url tag of aiohttp server spans lost the percent-encoding of the query string. The http.query.string tag now also keeps the percent-encoded query string.
• algoliasearch
  • This fix resolves an IndexError raised when collect_query_text is enabled and the search query is passed as a keyword argument.
  • This fix resolves an issue where applications failed to start when algoliasearch 4.0.0 or later was installed, raising TypeError: issubclass() arg 1 must be a class in Django applications.
• remote_config
  • Fixes an issue where updating or deleting one remote configuration for a service discards the tracing settings, such as sampling rules, tags, and log injection, applied by the other remote configurations that still target it.
  • Fixes an issue where forked worker processes, or processes that enable LLM Observability in agentless mode, keep applying a remote configuration that was removed while they were starting.
• code origin
  • Fixes an issue on Python 3.10 where an exception raised from an async generator decorated with tracer.wrap() would surface as TypeError.
• internal
  • Fixes an issue where forked child processes hang and never exit, causing memory usage to grow and the parent process to stall, when several threads fork at the same time (for example, multiprocessing.Process started from a thread pool).
  • Fixes an issue on Python 3.12 and earlier where a process can hang at exit instead of terminating when gevent is installed and remote configuration enables Exception Replay or Dynamic Instrumentation while the application is starting.
  • Fixes an issue where wrapping an obfuscated function (for example PyArmor) on Python 3.15 proceeds into instrumentation setup instead of failing cleanly. Those functions still cannot be instrumented.
  • Fixes an issue on Python 3.10 where inspecting the call stack from ddtrace instrumentation placed on async functions and generators could raise a TypeError, silently disabling that instrumentation for the call.
  • Fixes an issue where features that use sys.monitoring (such as Error Tracking handled-exception monitoring) could fail to enable with a ValueError on Python 3.12 and later when a garbage collection ran during startup.
• dynamic instrumentation
  • Fixes an issue where an application thread could hang indefinitely when probes, Exception Replay, or Code Origin produced data faster than it could be uploaded.
  • Fixes an issue where line probes placed on function or method declarations fail to install.
• elasticsearch
  • Fixes an issue where the Elasticsearch request duration metric is missing from spans when Elasticsearch reports a duration of zero milliseconds.
• falcon
  • Fixes an issue where creating a falcon.asgi.App failed with a TypeError or CompatibilityError when the falcon integration was enabled, for example under ddtrace-run. ASGI applications now start normally but are not traced by the falcon integration; wrap them with ddtrace.contrib.asgi.TraceMiddleware to trace them.
  • Fixes an issue where creating a falcon.App failed when middleware was None, a single component, a tuple, or passed positionally, and where falcon.API applications on Falcon 3 produced duplicate falcon.request spans.
  • Fixes an issue where a Falcon application returns a 500 Internal Server Error instead of the intended response when resp.status is set to an integer, an http.HTTPStatus member, or bytes, or when resp.status_code is set (Falcon 4).
• google-adk
  • Fixes an issue where applications using google-adk 2.10.0 and later fail to start when tracing is enabled, or start normally but report no tool call or code execution spans. Tool calls and code execution are now traced on all supported google-adk versions.
  • Fixes an issue where applications using google-adk 2.12.0 fail to start with LLM Observability enabled, or start without tool call and code execution spans otherwise. Tool calls made through Runner.run_live are now also traced on google-adk 2.12.0.
• graphql
  • Fixes an issue where queries fail with a TypeError when resolver tracing is enabled with graphql-core 3.2.10 or later. Restores resolver tracing for synchronous execution with graphql-core 3.3.
• LangChain
  • Fixes missing IAST command-injection findings in legacy agents provided by langchain-classic, and lost taint propagation when formatting chat prompts with LangChain v1.
  • Fixes AI Guard allowing legacy agent tools to execute before a blocking decision when using langchain-classic and the model response evaluation fails or the agent changes the tool arguments.
• IAST
  • Fixes missing vulnerability findings in response middleware around mounted web applications when nested request spans finish before the enclosing request.
  • Fixes an issue where background tasks or worker threads that outlive a request can affect vulnerability analysis for another request. Also prevents errors in instrumented regular-expression substitutions when a request finishes during taint propagation.
• Code Security (IAST)
  • Fixes a potential process crash when a tainted string subclass changes its taint state from its custom hash method.
  • Fixes an issue where SQL values escaped by PyMySQL 1.2.1 and later are not recognized as sanitized.
• LLM Observability
  • Fixes an issue where OpenAI and LiteLLM chat calls could block the application for seconds or longer when a chat message contained Action: followed by long content.
  • Fixes an issue where concurrently running OpenAI Agent workflows within the same traced operation can be missing from LLM Observability or report incorrect input/output.
  • Fixes an issue where retrieving Agent-delivered or cached prompts requires DD_API_KEY in the application. Fetching other prompts still requires the appropriate API credentials.
  • This fix resolves an issue where google_genai and vertexai LLM spans did not include tool_use_prompt_token_count in the input token count.
  • Fixes an issue where LLMJudge with the anthropic provider raised TypeError: Object of type method is not JSON serializable when the model returned a thinking block before the text block (for example, Claude models with adaptive thinking). The judge now reads the first text block in the response.
  • Fixes a regression introduced in 4.15.2 where LLM Observability spans that were still open when an ASGI request finished (FastAPI, Starlette, Django ASGI) were force-finished at request teardown. Agents and background tasks that intentionally outlive the HTTP request had their spans truncated, and work started afterwards was detached into separate root traces instead of staying nested. Abandoned streams are still reclaimed by TracedStream/TracedAsyncStream finalization and the stream context managers.
• otel
  • Fixes an issue where OTLP HTTP export requests are incorrectly traced when OpenTelemetry logs or metrics collection is enabled with opentelemetry-exporter-otlp 1.45.0 or later.
• opentelemetry
  • Fixes an issue where OTEL_LOGS_EXPORTER=otlp does not enable OTLP log export.
• telemetry
  • Fixes an issue where telemetry events can be associated with a previous AWS Lambda MicroVM invocation after the runtime identity is refreshed.
• vllm
  • Fixes an issue with vLLM >= 0.20.0 where the integration raised AttributeError: 'EngineCoreOutput' object has no attribute 'num_cached_tokens'. This error stopped the vLLM output loop and caused all requests to fail. The integration now also handles tracing errors without interrupting vLLM.
• kafka
  • Fixes an issue where Data Streams Monitoring did not link producers to consumers when Producer.produce() was called with headers as a list of tuples or as None.
• MCP
  • Server tool call spans now use the tool result's error text as the error message instead of a generic tool resulted in an error message.
  • Fixes an issue where failed server tool calls are not marked as errors when LLM Observability is disabled.
• pytest
  • Fixes deprecation warnings emitted by the Datadog pytest plugin when running test sessions with pluggy 1.7 or newer (for example, pytest 9.2).
• stats computation
  • restore stats computation status when re-enabling tracing after opting out. Opting out of APM tracing now also correctly disables stats computation.
• Fixes an issue where logging.captureWarnings(True) had no effect when running under ddtrace-run with gevent installed: warnings were written directly to stderr in the default plain-text format instead of being routed through the py.warnings logger.

Other Changes

• tracing
  • Reduces WSGI response iteration overhead. In an end-to-end traced Flask benchmark, many-chunk streaming responses are about 39% faster per request, while single-body responses are unaffected.