2026-11-05 Version 2.17.0
Breaking Changes
- Change the
rsssource configuration to afeedsmap and a new event schema as part of its rewrite; existingrsspipelines must update their configuration (#7002) - Stop acknowledging undelivered events in the CloudWatch Logs sink; failed batches now retry at most
max_retriestimes even without a DLQ and are negatively acknowledged when retries run out (#7211, #7212) - Stop copying
gen_ai.*fields and token totals onto non-GenAI root spans in theotel_tracesprocessor; custom queries or dashboards that read those fields on root spans will see different data, and settinggenai_root_propagation: all_rootsrestores the previous behavior (#7210)
Features
- Ingest data from existing Splunk forwarders and HTTP Event Collector (HEC) clients without any client changes using the new
splunk_hecsource, which supports the standard HEC endpoints, multiple tokens with per-token defaults, indexer acknowledgments, and backpressure (#6823) - Ingest multiple RSS and Atom feeds with the rewritten
rsssource, which deduplicates items, retries failed feeds independently with exponential backoff, and supports per-feed polling frequency and HTTP basic authentication (#7002) - Scale Data Prepper to multiple nodes using a relational database you already run, such as PostgreSQL or MySQL, with the new
jdbcsource coordination store (#6740) - Add an opt-in plugin framework backed by an embedded Apache Felix OSGi runtime (#6760)
Enhancements
- Improve load balancing for horizontally scaled OpenTelemetry sources with new gRPC
max_connection_ageandconnection_drain_durationsettings that make clients periodically reconnect and discover new replicas (#6745) - Support Amazon OpenSearch Service domains optimized for observability with the
trace-analytics-plain-raw-v2index type in the OpenSearch sink (#7188) - Support string values up to 64 MB in events, up from the previous 20 MB limit (#7145)
- Support
azure_federatedOAuth authentication for Azure Event Hubs in the Kafka source (#7011) - Support Map, List, and Array types in the
length()expression function (#7030) - Attach a per-instance
instance_labelin the Prometheus sink so multiple Data Prepper nodes can safely write the same series (#7023) - Write to Amazon OpenSearch Serverless time series collections by automatically setting the timestamp field in the OpenSearch sink (#7221)
- Support a dynamic per-event entity in the CloudWatch Logs sink (#7032)
- Add a
sumaction to theaggregateprocessor to sum a numeric field per group (#7024) - Support custom rate limits by group in the
aggregateprocessorrate_limiteraction (#7096) - Support
metadata_max_age,acknowledgments_timeout, andacknowledgments_expiry_resetoptions in the Kafka source (#7073) - Support
connections_max_idlein the Kafka source (#7016) - Automatically detect and skip pre-compression JSON metadata headers when decompressing files (#7021)
- Make the DynamoDB source stream shard discovery interval configurable (#7000)
- Make the acknowledgements shutdown timeout configurable in
data-prepper-config.yaml(#7127)
Bug Fixes
- Fix the Jira source failing to crawl after Atlassian removed the
/rest/api/3/searchendpoint (#7104, #7143) - Fix the
otel_trace_sourcedropping the entire OTLP request when a span has duplicate attribute keys (#7151) - Fix the RDS source dropping UPDATE events that set a column to NULL on a cascading foreign key parent table (#7129)
- Fix the Kafka source
azure_federatedauthentication stopping after the first OAuth token refresh (#7039) - Fix the pull-based ingestion routing hash to match OpenSearch shard routing (#6993)
- Log the OpenSearch sink first-write-wins notice once per data stream instead of once per document (#6972)
- Parse top-level JSON arrays in the
parse_jsonandparse_ionprocessors (#5705) - Fix handling of JSON pointer escape sequences when setting event keys (#6150)
- Perform per-field conflict resolution with
overwrite_if_destination_existswhen thekey_valueprocessor writes to a destination (#7091)
Security
- Pass the PostgreSQL database name as a driver property in the RDS source to prevent a pipeline author from injecting connection string parameters that could exfiltrate database credentials. Fixes GHSA-h88q-pr3p-23cj. (#7255)
- Bypass class name resolution in Glue Schema Registry JSON deserialization to prevent unvalidated class loading. Fixes GHSA-c5wf-7mjx-mcx6. (#7256)
- CVE-2026-59890 - setuptools 83 (#7009)
- CVE-2026-102598 - werkzeug 3.1.9 (#7176)
Maintenance
- Make the
dissectprocessor thread-safe (#5546)