opensearch-project/OpenSearch 3.4.0

on GitHub

Version 3.4.0 Release Notes

Compatible with OpenSearch and OpenSearch Dashboards version 3.4.0

Added

• Allow setting index.creation_date on index creation and restore for plugin compatibility and migrations (#19931)
• Add support for a ForkJoinPool type (#19008)
• Add seperate shard limit validation for local and remote indices (#19532)
• Use Lucene pack method for half_float and unsigned_long when using ApproximatePointRangeQuery (#19553)
• New cluster setting search.query.max_query_string_length_monitor_only (#19539)
• Add a mapper for context aware segments grouping criteria (#19233)
• Return full error for GRPC error response (#19568)
• Add support for repository with Server side encryption enabled and client side encryption as well based on a flag (#19630)
• Add pluggable gRPC interceptors with explicit ordering(#19005)
• Add BindableServices extension point to transport-grpc-spi (#19304)
• Add metrics for the merged segment warmer feature (#18929)
• Handle deleted documents for filter rewrite sub-aggregation optimization (#19643)
• Add bulk collect API for filter rewrite sub-aggregation optimization (#19933)
• Allow collectors take advantage of preaggregated data using collectRange API (#20009)
• Bulk collection logic for metrics and cardinality aggregations (#20067)
• Add pointer based lag metric in pull-based ingestion (#19635)
• Introduced internal API for retrieving metadata about requested indices from transport actions (#18523)
• Add cluster defaults for merge autoThrottle, maxMergeThreads, and maxMergeCount; Add segment size filter to the merged segment warmer (#19629)
• Add build-tooling to run in FIPS environment (#18921)
• Add SMILE/CBOR/YAML document format support to Bulk GRPC endpoint (#19744)
• Make test-suite runnable under FIPS compliance support (#18491)
• Implement GRPC Search params Highlightand Sort (#19868)
• Implement GRPC ConstantScoreQuery, FuzzyQuery, MatchBoolPrefixQuery, MatchPhrasePrefix, PrefixQuery, MatchQuery (#19854)
• Add async periodic flush task support for pull-based ingestion (#19878)
• Add support for context aware segments (#19098)
• Implement GRPC FunctionScoreQuery (#19888)
• Implement error_trace parameter for bulk requests (#19985)
• Allow the truncate filter in normalizers (#19778)
• Support pull-based ingestion message mappers and raw payload support (#19765)
• Add search API tracker (#18601)
• Support dynamic consumer configuration update in pull-based ingestion (#19963)
• Cache the StoredFieldsReader for scroll query optimization (#20112)
• Add Hybrid Cardinality collector to prioritize Ordinals Collector (#19524)

Changed

• Combining filter rewrite and skip list to optimize sub aggregation(#19573)
• Faster terms query creation for keyword field with index and docValues enabled (#19350)
• Refactor to move prepareIndex and prepareDelete methods to Engine class (#19551)
• Omit maxScoreCollector in SimpleTopDocsCollectorContext when concurrent segment search enabled (#19584)
• Onboarding new maven snapshots publishing to s3 (#19619)
• Remove MultiCollectorWrapper and use MultiCollector in Lucene instead (#19595)
• Change implementation for percentiles aggregation for latency improvement (#19648)
• Wrap checked exceptions in painless.DefBootstrap to support JDK-25 (#19706)
• Refactor the ThreadPoolStats.Stats class to use the Builder pattern instead of constructors (#19317)
• Refactor the IndexingStats.Stats class to use the Builder pattern instead of constructors (#19306)
• Remove FeatureFlag.MERGED_SEGMENT_WARMER_EXPERIMENTAL_FLAG. (#19715)
• Replace java.security.AccessController with org.opensearch.secure_sm.AccessController in sub projects with SocketAccess class (#19803)
• Replace java.security.AccessController with org.opensearch.secure_sm.AccessController in discovery plugins (#19802)
• Change the default value of doc_values in WildcardFieldMapper to true. (#19796)
• Make Engine#loadHistoryUUID() protected and Origin#isFromTranslog() public (#19753)
• Bump opensearch-protobufs dependency to 0.23.0 and update transport-grpc module compatibility (#19831)
• Refactor the RefreshStats class to use the Builder pattern instead of constructors (#19835)
• Refactor the DocStats and StoreStats class to use the Builder pattern instead of constructors (#19863)
• Pass registry of headers from ActionPlugin.getRestHeaders to ActionPlugin.getRestHandlerWrapper (#19875)
• Refactor the Condition.Stats and DirectoryFileTransferTracker.Stats class to use the Builder pattern instead of constructors (#19862)
• Refactor the RemoteTranslogTransferTracker.Stats and RemoteSegmentTransferTracker.Stats class to use the Builder pattern instead of constructors (#19837)
• Refactor the GetStats, FlushStats and QueryCacheStats class to use the Builder pattern instead of constructors (#19935)
• Add RangeSemver for dependencies in plugin-descriptor.properties (#19939)
• Refactor the FieldDataStats and CompletionStats class to use the Builder pattern instead of constructors (#19936)
• Thread Context Preservation by gRPC Interceptor (#19776)
• Update NoOpResult constructors in the Engine to be public (#19950)
• Make XContentMapValues.filter case insensitive (#19976)
• Refactor the TranslogStats and RequestCacheStats class to use the Builder pattern instead of constructors (#19961)
• Improve performance of matrix_stats aggregation (#19989)
• Refactor the IndexPressutreStats, DeviceStats and TransportStats class to use the Builder pattern instead of constructors (#19991)
• Refactor the Cache.CacheStats class to use the Builder pattern instead of constructors (#20015)
• Refactor the HttpStats, ScriptStats, AdaptiveSelectionStats and OsStats class to use the Builder pattern instead of constructors (#20014)
• Bump opensearch-protobufs dependency to 0.24.0 and update transport-grpc module compatibility (#20059)
• Refactor the ShardStats, WarmerStats and IndexingPressureStats class to use the Builder pattern instead of constructors (#19966)
• Add skiplist optimization to auto_date_histogram aggregation (#20057)
• Throw exceptions for currently unsupported GRPC request-side fields (#20162)

Fixed

• Fix Allocation and Rebalance Constraints of WeightFunction are incorrectly reset (#19012)
• Fix flaky test FieldDataLoadingIT.testIndicesFieldDataCacheSizeSetting (#19571)
• Fix case-insensitive wildcard + aggregation query crash (#19489)
• Avoid primary shard failure caused by merged segment warmer exceptions (#19436)
• Fix pull-based ingestion out-of-bounds offset scenarios and remove persisted offsets (#19607)
• Fix issue with updating core with a patch number other than 0 (#19377)
• [Java Agent] Allow JRT protocol URLs in protection domain extraction (#19683)
• Fix potential concurrent modification exception when updating allocation filters (#19701)
• Fix wildcard query with escaped backslash followed by wildcard character (#19719)
• Fix file-based ingestion consumer to handle start point beyond max line number (#19757)
• Fix IndexOutOfBoundsException when running include/exclude on non-existent prefix in terms aggregations (#19637)
• Fixed assertion unsafe use of ClusterService.state() in ResourceUsageCollectorService ([#19775])(#19775))
• Fix Unified highlighter for nested fields when using matchPhrasePrefixQuery (#19442)
• Add S3Repository.LEGACY_MD5_CHECKSUM_CALCULATION to list of repository-s3 settings (#19788)
• Fix NullPointerException when restoring remote snapshot with missing shard size information (#19684)
• Fix NPE of ScriptScoreQuery (#19650)
• Fix ClassCastException in FlightClientChannel for requests larger than 16KB (#20010)
• Fix GRPC Bulk (#19937)
• Fix node bootstrap error when enable stream transport and remote cluster state (#19948)
• Keep track and release Reactor Netty 4 Transport accepted Http Channels during the Node shutdown (#20106)
• Fix deletion failure/error of unused index template; case when an index template matches a data stream but has a lower priority. (#20102)
• Fixed version incompatibility in remote state entities using bytestream for ser/de (#20080)
• Fix toBuilder method in EngineConfig to include mergedSegmentTransferTracker(#20105)
• Fixed handling of property index in BulkRequest during deserialization (#20132)
• Fix negative CPU usage values in node stats (#19120)
• Fix duplicate registration of FieldDataCache dynamic setting (#20140)
• Fix array out of bounds during aggregation (#20204)

Dependencies

• Bump Apache Lucene from 10.3.1 to 10.3.2 (#20026)
• Update to Gradle 9.2 (#19575) (#19856)
• Update bundled JDK to JDK-25 (#19314)
• Bump peter-evans/create-or-update-comment from 4 to 5 (#19536)
• Bump com.azure:azure-core-http-netty from 1.15.12 to 1.16.1 (#19533)
• Bump org.apache.zookeeper:zookeeper from 3.9.3 to 3.9.4 (#19535)
• Bump com.azure:azure-storage-common from 12.30.2 to 12.30.3 (#19615)
• Bump peter-evans/create-issue-from-file from 5 to 6 (#19616)
• Bump com.squareup.okhttp3:okhttp from 5.1.0 to 5.3.0 (#19614, #19944)
• Bump com.microsoft.azure:msal4j from 1.21.0 to 1.23.1 (#19688)
• Bump commons-net:commons-net from 3.11.1 to 3.12.0 (#19687)
• Bump org.apache.avro:avro from 1.12.0 to 1.12.1 (#19692)
• Bump com.github.spotbugs:spotbugs-annotations from 4.9.6 to 4.9.8 (#19691)
• Bump stefanzweifel/git-auto-commit-action from 6 to 7 (#19689)
• Bump ch.qos.logback modules from 1.5.18 to 1.5.20 in HDFS test fixture (#19763)
• Bump github/codeql-action from 3 to 4 (#19785)
• Bump gradle/actions from 4 to 5 (#19781)
• Bump org.bouncycastle:bc-fips from 2.1.1 to 2.1.2 (#19818)
• Bump com.google.http-client:google-http-client-gson from 1.47.1 to 2.0.0 (#19253)
• Bump com.sun.xml.bind:jaxb-impl from 2.2.3-1 to 4.0.6 (#19472)
• Bump tj-actions/changed-files from 46.0.5 to 47.0.0 (#19873)
• Bump com.google.api.grpc:proto-google-iam-v1 from 1.55.0 to 1.57.0 (#19872)
• Bump org.apache.commons:commons-text from 1.13.1 to 1.14.0 (#19871)
• Exclude group com.microsoft.sqlserver from hadoop-minicluster (#19889)
• Bump netty to 4.2.4 (#19178)
• Bump actions/github-script from 7 to 8 (#19946)
• Bump com.google.api:gax-httpjson from 2.69.0 to 2.72.1 (#19943)
• Update Hadoop to 3.4.2 and enable security (Kerberos) integration tests under JDK-24 and above (#19952)
• Bump com.google.cloud:google-cloud-storage from 2.55.0 to 2.60.0 (#20023)
• Bump commons-cli:commons-cli from 1.10.0 to 1.11.0 (#20022)
• Bump com.squareup.okio:okio from 3.16.0 to 3.16.3 (#20025)
• Bump org.tukaani:xz from 1.10 to 1.11 (#20082)
• Bump com.google.api:api-common from 2.52.0 to 2.55.1 (#20083)
• Bump actions/upload-artifact from 4 to 5 (#20081)
• Bump com.nimbusds:nimbus-jose-jwt from 10.5 to 10.6 (#20084)
• Bump commons-logging from 1.2 to 1.3.5 (#20156)

Deprecated

• Deprecated existing constructors in ThreadPoolStats.Stats in favor of the new Builder (#19317)
• Deprecated existing constructors in IndexingStats.Stats in favor of the new Builder (#19306)
• Deprecated existing constructors in RefreshStats in favor of the new Builder (#19835)
• Deprecated existing constructors in DocStats and StoreStats in favor of the new Builder (#19863)
• Deprecated existing constructors in Condition.Stats and DirectoryFileTransferTracker.Stats in favor of the new Builder (#19862)
• Deprecated existing constructors in RemoteTranslogTransferTracker.Stats and RemoteSegmentTransferTracker.Stats in favor of the new Builder (#19837)
• Deprecated existing constructors in GetStats, FlushStats and QueryCacheStats in favor of the new Builder (#19935)
• Deprecated existing constructors in FieldDataStats and CompletionStats in favor of the new Builder (#19936)
• Deprecated existing constructors in TranslogStats and RequestCacheStats in favor of the new Builder (#19961)
• Deprecated existing constructors in IndexPressutreStats, DeviceStats and TransportStats in favor of the new Builder (#19991)
• Deprecated existing constructors in Cache.CacheStats in favor of the new Builder (#20015)
• Deprecated existing constructors in HttpStats, ScriptStats, AdaptiveSelectionStats and OsStats in favor of the new Builder (#20014)
• Deprecated existing constructors in ShardStats, WarmerStats and IndexingPressureStats in favor of the new Builder (#19966)