DataDog/dd-trace-go v2.10.0

on GitHub

Summary

This release introduces the new Efficient Trace Protocol (ETP; v1.0) by default. If you want to opt-out and continue setting DD_TRACE_AGENT_PROTOCOL_VERSION=0.4. ETP introduces significant improvements on performance and payload size in the wire.

There are other improvements regarding allocations and CPU usage in our tracer's hot paths, including an experimental span pool (available as tracer option and environment variable DD_TRACER_EXPERIMENTAL_SPAN_POOL_ENABLED).

New contribs:

• labstack/echo v5
• aerospike/aerospike-client-go v7

What's Changed

Application Performance Monitoring (APM)

• chore: code improvements to support testing for ETP by @hannahkm in #4719
• test(instrumentation/httptrace): cover query-string obfuscator in tracked benches by @darccio in #4806
• feat: trace context propagation behavior by @igoragoli in #4635
• fix(contrib/aws/aws-sdk-go-v2): fix EventBridge DSM tag structure by @robcarlan-datadog in #4832
• feat(ddtrace/tracer): add span pool by @darccio in #4440
• feat(tracer/meter): emit otel Go runtime metrics by @anna-git in #4803
• feat(contrib): add labstack/echo v5 integration by @genesor in #4648
• feat(contrib): add integration for aerospike/aerospike-client-go.v7 by @darccio in #4830
• perf: multiple improvements backport [v2.10.0] by @darccio in #5019
• fix(internal/orchestrion): backport GLS pool/parent fixes to v2.10.x by @darccio in #5097
• chore: backport multiple PRs to release-v2.10.x by @darccio in #5174
• feat(tracer): remove DD_TRACE_CLIENT_HOSTNAME_COMPAT by @mtoffl01 in #4758
• refactor(config): migrate dynamicInstrumentationEnabled by @mtoffl01 in #4760
• refactor(config): migrate traceAnalyticsEnabled by @mtoffl01 in #4795
• refactor(config): migrate OTEL_LOGS_EXPORTER logic into loadConfig by @MilanGarnier in #4790
• refactor(config): migrate ciVisibilityAgentless by @mtoffl01 in #4793
• fix(tracing): do not keep credential metadata in gRPC spans by @KowalskiThomas in #4767
• test(ddtrace/tracer): add harness tests for removing SpanContext's span back-reference by @darccio in #4815
• refactor(config): migrate ciVisibilityNoopTracer by @mtoffl01 in #4819
• refactor(ddtrace/tracer): remove SpanContext.span back-reference by @darccio in #4816
• feat(instrumentation/httptrace): optimize default behaviour for URL query args' obfuscation by @darccio in #4805
• feat(contrib/jackc/pgx.v5): add WithPoolName option for pool name tagging by @marstid in #4748
• test(ddtrace/tracer): stabilize span pool agent POV's flaky tests by @darccio in #4850
• feat(config): Add DD_TRACE_AGENT_TIMEOUT config option by @Seanbon0611 in #4827
• refactor(config): migrate sendRetries to internal config by @eunskin in #4743
• feat(config): migrate universalVersion, add DD_TRACE_UNIVERSAL_VERSION_ENABLED by @mtoffl01 in #4820
• feat(config): migrate dogstatsdAddr, add DD_DOGSTATSD_URL by @mtoffl01 in #4796
• fix(css): Populate RuntimeID and Service on CSS (APMSP-3042) by @ichinaski in #4875
• feat(config): migrate headerAsTags, add apply-callback to DynamicConfig by @mtoffl01 in #4821
• docs(contrib/labstack/echo.v4): fix inverted WithErrorCheck doc comment by @korECM in #4858
• fix(ddtrace/tracer): recover from stale-idle UDS conn races by @kakkoyun in #4809
• chore(telemetry): exclude OTLP headers from configuration telemetry by @bm1549 in #4859
• feat(ddtrace/tracer): add span lifecycle benchmarks by @darccio in #4682
• fix(instrumentation/httptrace): fix O(N²) CPU exhaustion in query-string obfuscator by @darccio in #4896
• fix(telemetry): derive agentless telemetry URL from DD_SITE by @Nitjsefni7 in #4883
• fix(ddtrace/tracer): classify ENOTCONN as transient and raise stale-conn retry budget by @darccio in #4908
• perf(ddtrace/tracer): remove redundant per-trace process-tags write in setTraceTagsLocked by @darccio in #4773
• refactor(config): migrate globalTags by @funyjane in #4865
• test(ddtrace/tracer): fix flaky TestHighPrecisionTimerIsMoreAccurate on Windows by @darccio in #4910
• feat(orchestrion): weave the full GLS lifecycle (tracer & dyngo stay GLS-free) (alt to #4808) by @kakkoyun in #4891
• fix(ddtrace/tracer): avoid nested same-class span locks in partial flush by @darccio in #4909
• fix(ddtrace/tracer): skip TestPartialFlushSpanLockOrderingCycle by @darccio in #4922
• test(ddtrace/tracer): de-flake and re-enable TestPartialFlushSpanLockOrderingCycle by @darccio in #4924
• fix(opentelemetry/metric): honor configured temporality selector by @kakkoyun in #4847
• fix(ddtrace/tracer): make traceID hex cache safe for concurrent reads by @darccio in #4726
• fix(tracer): export OTel semconv integer attributes as int in OTLP by @link04 in #4888
• chore: enable ETP by default by @hannahkm in #4849
• fix(internal/remoteconfig): poll immediately on Subscribe by @aarsilv in #4958
• fix(appsec): use bit shift instead of XOR for map presize hints by @eliottness in #4709
• fix(tracer): don't activate runtime metrics v1 if otel runtime metrics are enabled by @anna-git in #4962
• feat(tracer): add WithStatsdClient option for sharing statsd clients (#4935) [backport] by @darccio in #4992
• feat(ddtrace/tracer): adaptive pre-grow for payloadV04 and payloadV1 buffers (#4981) [backport] by @darccio in #4996
• perf: backport #5039 and #5040 to v2.10.x by @kakkoyun in #5044

Application and API Protection (AAP)

• perf(appsec): make rate limiter synchronous to remove idle goroutine cost by @eliottness in #4956
• feat(aap): collect security testing headers by @e-n-0 in #4774
• fix(contrib/azure/apim-callout): bound inline body decoding to prevent OOM by @eliottness in #4797
• feat(appsec): upgrade to go-libddwaf v5 with subcontext-based ephemeral runs by @eliottness in #4938
• fix(contrib/azure/apim-callout): restore AppSec body inspection on oversized inline bodies and Boomi block enforcement by @eliottness in #4966
• chore(appsec): backport go-libddwaf v5.0.0 (#4985) + CMDi RASP (#4978) to release-v2.10.x by @eliottness in #4986
• fix(appsec): plug GLS + dataBroadcaster listener leak in ServiceEntrySpanOperation by @aqaurius6666 in #4766
• test(internal/appsec/apisec): reduce sampler workload to prevent CI timeout by @darccio in #4912

CI Visibility

• fix(civisibility): propagate Jenkins custom parent ID by @juan-fernandez in #4750
• fix(internal/civisibility): fix known-tests API pagination by @anmarchenko in #4824
• fix(internal/civisibility): fix agent UDS requests by @tonyredondo in #4852
• fix(civisibility): truncate CI Visibility meta values by @tonyredondo in #4853
• feat(ci-visibility): cache Go read requests and add git upload kill switch by @tonyredondo in #4786
• feat(civisibility): backfill coverage for ITR skips by @tonyredondo in #4862
• fix(civisibility): prevent test process crash on transient settings fetch failure by @darccio in #4905
• fix(civisibility): retry JSON-expected 2xx responses with unrecognised Content-Type; harden subtest matrix harness by @darccio in #4917
• fix(civisibility): prevent syncFeatureMetadataFromExecution from corrupting management directives across ATF retries by @darccio in #4914
• feat(civisibility): upload code coverage reports by @tonyredondo in #4918

Database Monitoring (DBM)

• feat(contrib/database/sql): Propagate base hash on DBM comment and introduce dynamic_service mode by @amarziali in #4907

Data Stream Monitoring (DSM)

• perf(datastreams): reduce per-message allocations and lock contention in DSM hot path by @robcarlan-datadog in #4920
• feat(contrib/aws/aws-sdk-go-v2): add DSM propagation for EventBridge, SQS, SNS, and Kinesis by @jeastham1993 in #4772
• fix(contrib/aws/aws-sdk-go-v2): update eventbridge dsm checkpoint tags by @jeastham1993 in #4867

Feature Flagging & Experimentation (FFE)

• perf(openfeature): add evaluation benchmarks to CI by @leoromanovsky in #4902
• feat(openfeature): server-side EVP flagevaluation emission by @leoromanovsky in #4886
• refactor(openfeature): disambiguate eval metrics and logging hooks by @dd-oleksii in #4937
• fix(openfeature): isolate invalid flags [backport v2.10] by @leoromanovsky in #5104

LLMObs

• feat(llmobs): propagate experiment project_id via baggage by @rarguelloF in #4801
• feat(llmobs): add WithAnnotatedCostTagKeys by @XG-xin in #4728
• feat(llmobs): add MetricKey constants for cache breakdown and other token metrics by @rarguelloF in #4784
• feat(llmobs): add tool_version to ToolDefinition and thread it to child tool spans by @imran-at-datadog in #4818
• feat(llmobs/experiment): add EvaluatorResult for reasoning, assessment, metadata, and tags by @hemeryar in #4825
• feat(llmobs/dataset): add WithPullVersion option for historical dataset snapshots by @DD-youssef-ahmed in #4814

Profiling

• chore(profiler): Set EVP origin headers on profile uploads by @felixge in #4898

Serverless

• feat(contrib/aws/datadog-lambda-go): disable internal statsd metrics in Lambda by @purple4reina in #4841

General

• ci: backport Go 1.27/1.26.5 CI stability fixes from main by @darccio in #5264
• fix(deps): bump go.opentelemetry.io/otel/* to non-vulnerable versions by @darccio in #4810
• chore: update vulnerabilities by @hannahkm in #4831
• fix: bump dependencies to cover multiple third-parties' CVEs by @darccio in #4906
• chore: bump go-libddwaf/v4 from v4.9.0 to v4.10.0 across all modules by @darccio in #4913
• fix: clean up github.com/armon/go-metrics warnings and bump github.com/99designs/gqlgen to v0.17.92 by @darccio in #4916
• chore(deps): bump the docker group across 6 directories with 3 updates by @dependabot[bot] in #4837
• fix(deps): bump klauspost/compress to v1.18.6 by @darccio in #4751
• chore: upgrade golang.org/x/net to v0.53.0 and google.golang.org/grpc to v1.79.3 by @hannahkm in #4755
• feat: add AGENTS files by @hannahkm in #4674
• chore: bump github.com/DataDog/datadog-agent/pkg/* to v0.79.0 by @darccio in #4787
• ci: sandbox govulncheck and latest-dependency smoke tests by @kakkoyun in #4598
• chore(ci): replace secrets.GITHUB_TOKEN with dd-octo-sts by @lloeki in #4749
• chore: improve payload benchmarking by @hannahkm in #4688
• feat(autoreleasetagger): single-commit multi-tag model with safety guards and CI integration tests by @darccio in #4764
• fix(ci): use GITHUB_TOKEN for GHCR image publishing by @e-n-0 in #4777
• feat: reenable Dependabot for GHA and Dockerfiles with cooldown by @darccio in #4705
• fix(autoreleasetagger): honor excluded modules in tag guard by @darccio in #4778
• fix(.github/chainguard): add workflows permission to sync-cue-actions by @darccio in #4780
• feat: support container tags hash in DSM by @raphaelgavache in #4759
• ci(gitlab): reorder config-validation stage in CI pipeline by @genesor in #4804
• fix(ci): ignore devflow/mergegate in all-jobs-are-green by @rarguelloF in #4807
• fix(testutils): acquire trace mutex in SetPropagatingTag to prevent concurrent map writes by @darccio in #4833
• feat: improve performance and docs on existing state machines by @darccio in #4811
• chore(chainguard): add dd-octo-sts policy for FPD config-inversion by @genesor in #4856
• chore: update push to test optim version by @hannahkm in #4861
• refactor(config): migrate orchestrionCfg by @funyjane in #4826
• chore(ci): fix codeowner resolution by @cbeauchesne in #4863
• feat(ci): add build metrics tracking for standard vs Orchestrion builds by @darccio in #4877
• feat(ci): expand build-metrics workflow with all integrations and fix triggers by @darccio in #4878
• feat(ci): send all build duration samples instead of median by @darccio in #4881
• ci: add TRACE_STATS_COMPUTATION to system-tests matrix by @ichinaski in #4882
• test(internal/appsec/apisec): fix flaky/broken API Sec sampler test on gotip by @RomainMuller in #4864
• docs(agents): add comment style guideline to CONTRIBUTING.md by @mtoffl01 in #4884
• ci(system-tests): pre-build weblog images once per variant and cache datadog-ci npm install to eliminate transient failures by @darccio in #4903
• chore(deps): bump the github-actions group across 3 directories with 18 updates by @dependabot[bot] in #4915
• chore: align Datadog Code Coverage uploads with Codecov (contrib, gotip, smoke-tests, core) by @nikita-tkachenko-datadog in #4923
• chore: remove CodeCov upload from CI by @nikita-tkachenko-datadog in #4901
• chore(deps): bump the github-actions group across 3 directories with 3 updates by @dependabot[bot] in #4955
• chore(deps): bump the github-actions group across 3 directories with 4 updates by @dependabot[bot] in #4961
• chore(scripts): add configaudit tool for tracking config migration progress by @bm1549 in #4794

New Contributors

• @aqaurius6666 made their first contribution in #4766
• @XG-xin made their first contribution in #4728
• @MilanGarnier made their first contribution in #4790
• @imran-at-datadog made their first contribution in #4818
• @KowalskiThomas made their first contribution in #4767
• @hemeryar made their first contribution in #4825
• @DD-youssef-ahmed made their first contribution in #4814
• @Seanbon0611 made their first contribution in #4827
• @anna-git made their first contribution in #4803
• @Nitjsefni7 made their first contribution in #4883
• @nikita-tkachenko-datadog made their first contribution in #4923
• @aarsilv made their first contribution in #4958

Full Changelog: v2.9.2...v2.10.0