opensearch-project/OpenSearch 2.9.0

on GitHub

2023-07-13 Version 2.9.0 Release Notes

[2.9]

Added

• [SearchPipeline] Add new search pipeline processor type, SearchPhaseResultsProcessor, that can modify the result of one search phase before starting the next phase.(#7283)
• Add task cancellation monitoring service (#7642)
• Add TokenManager Interface (#7452)
• Add Remote store as a segment replication source (#7653)
• Implement concurrent aggregations support without profile option (#7514)
• Add dynamic index and cluster setting for concurrent segment search (#7956)
• Add descending order search optimization through reverse segment read. (#7967)
• [Search pipelines] Added search pipelines output to node stats (#8053)
• Update components of segrep backpressure to support remote store. (#8020)
• Make remote cluster connection setup in async (#8038)
• Add API to initialize extensions (#8029#8029)
• Add distributed tracing framework (#7543)
• Enable Point based optimization for custom comparators (#8168)
• [Extensions] Support extension additional settings with extension REST initialization (#8414)
• Adds mock implementation for TelemetryPlugin (#7545)
• Support transport action names when registering NamedRoutes (#7957)
• Create concept of persistent ThreadContext headers that are unstashable (#8291#8291)
• [Search pipelines] Add Global Ignore_failure options for Processors (#8373)
• Enable Partial Flat Object (#7997)
• Add jdk.incubator.vector module support for JDK 20+ (#8601)
• Introduce full support for Search Pipeline (#8613)
• Add partial results support for concurrent segment search (#8306)

Dependencies

• Bump com.azure:azure-storage-common from 12.21.0 to 12.21.1 (#7566, #7814)
• Bump com.google.guava:guava from 30.1.1-jre to 32.1.1-jre (#7565, #7811, #7807, #7808, #8402, #8400, #8401, #8581)
• Bump net.minidev:json-smart from 2.4.10 to 2.4.11 (#7660, #7812)
• Bump org.gradle.test-retry from 1.5.2 to 1.5.3 (#7810)
• Bump com.diffplug.spotless from 6.17.0 to 6.18.0 (#7896)
• Bump jackson from 2.15.1 to 2.15.2 (#7897)
• Add com.github.luben:zstd-jni version 1.5.5-3 (#2996)
• Bump netty from 4.1.91.Final to 4.1.93.Final (#7901)
• Bump com.amazonaws 1.12.270 to software.amazon.awssdk 2.20.55 (7372)
• Add org.reactivestreams 1.0.4 (7372)
• Bump com.networknt:json-schema-validator from 1.0.81 to 1.0.85 ([7968], #8255)
• Bump com.netflix.nebula:gradle-extra-configurations-plugin from 9.0.0 to 10.0.0 in /buildSrc (#7068)
• Bump com.google.guava:guava from 32.0.0-jre to 32.0.1-jre (#8009)
• Bump commons-io:commons-io from 2.12.0 to 2.13.0 (#8014, #8013, #8010)
• Bump com.diffplug.spotless from 6.18.0 to 6.19.0 (#8007)
• Bump 'com.azure:azure-storage-blob to 12.22.2 from 12.21.1 (#8043)
• Bump org.jruby.joni:joni from 2.1.48 to 2.2.1 (#8015, #8254)
• Bump com.google.guava:guava from 32.0.0-jre to 32.0.1-jre (#8011, #8012, #8107)
• Bump io.projectreactor:reactor-core from 3.4.18 to 3.5.6 in /plugins/repository-azure (#8016)
• Bump spock-core from 2.1-groovy-3.0 to 2.3-groovy-3.0 (#8122)
• Bump com.networknt:json-schema-validator from 1.0.83 to 1.0.84 (#8141)
• Bump com.netflix.nebula:gradle-info-plugin from 12.1.3 to 12.1.5 (#8139, #8568)
• Bump commons-io:commons-io from 2.12.0 to 2.13.0 in /plugins/discovery-azure-classic (#8140)
• Bump mockito from 5.2.0 to 5.4.0 (#8181)
• Bump netty from 4.1.93.Final to 4.1.94.Final (#8191)
• Bump org.apache.hadoop:hadoop-minicluster from 3.3.5 to 3.3.6 (#8257)
• Bump io.projectreactor.netty:reactor-netty-http from 1.1.7 to 1.1.8 (#8256)
• [Upgrade] Lucene 9.7.0 release (#8272)
• Bump org.jboss.resteasy:resteasy-jackson2-provider from 3.0.26.Final to 6.2.4.Final in /qa/wildfly (#8209)
• Bump com.google.api-client:google-api-client from 1.34.0 to 2.2.0 (#8276)
• Update Apache HttpCore/ HttpClient and Apache HttpCore5 / HttpClient5 dependencies (#8434)
• Bump org.apache.maven:maven-model from 3.9.2 to 3.9.3 (#8403)
• Bump io.projectreactor.netty:reactor-netty and io.projectreactor.netty:reactor-netty-core from 1.1.7 to 1.1.8 (#8405)
• Bump com.azure:azure-storage-blob from 12.22.2 to 12.22.3 (#8572)
• Bump net.minidev:json-smart from 2.4.11 to 2.5.0 (#8575, #8576)
• Bump com.google.jimfs:jimfs from 1.2 to 1.3.0 (#8577, #8571)
• Bump com.networknt:json-schema-validator from 1.0.85 to 1.0.86 (#8573)
• Bump com.google.cloud:google-cloud-core-http from 2.17.0 to 2.21.0 (#8586)
• Bump com.google.jimfs:jimfs from 1.2 to 1.3.0 (#8585)

Changed

• Replace jboss-annotations-api_1.2_spec with jakarta.annotation-api (#7836)
• Reduce memory copy in zstd compression (#7681)
• Add min, max, average and thread info to resource stats in tasks API (#7673)
• Add ZSTD compression for snapshotting (#2996)
• Change com.amazonaws.sdk.ec2MetadataServiceEndpointOverride to aws.ec2MetadataServiceEndpoint (7372)
• Change com.amazonaws.sdk.stsEndpointOverride to aws.stsEndpointOverride (7372)
• Add new query profile collector fields with concurrent search execution (#7898)
• Align range and default value for deletes_pct_allowed in merge policy (#7730)
• Rename QueryPhase actors like Suggest, Rescore to be processors rather than phase (#8025)
• Compress and cache cluster state during validate join request (#7321)
• [Snapshot Interop] Add Changes in Create Snapshot Flow for remote store interoperability. (#7118)
• Allow insecure string settings to warn-log usage and advise to migration of a newer secure variant (#5496)
• Add self-organizing hash table to improve the performance of bucket aggregations (#7652)
• Check UTF16 string size before converting to String to avoid OOME (#7963)
• Move ZSTD compression codecs out of the sandbox (#7908)
• Update ZSTD default compression level (#8471)
• [Search Pipelines] Pass pipeline creation context to processor factories (#8164)
• Enabling compression levels for zstd and zstd_no_dict (#8312)
• Optimize Metadata build() to skip redundant computations as part of ClusterState build (#7853)
• Add safeguard limits for file cache during node level allocation (#8208)
• Move span actions to Scope (#8411)
• Add wrapper tracer implementation (#8565)
• Improved performance of parsing floating point numbers (#7909)

Removed

• Remove COMPRESSOR variable from CompressorFactory and use DEFLATE_COMPRESSOR instead (7907)
• Remove concurrency based minimum file cache size restriction (#8294)

Fixed

• Fixing error: adding a new/forgotten parameter to the configuration for checking the config on startup in plugins/repository-s3 #7924
• Enforce 512 byte document ID limit in bulk updates (#8039)
• With only GlobalAggregation in request causes unnecessary wrapping with MultiCollector (#8125)
• Fix mapping char_filter when mapping a hashtag (#7591)
• Fix NPE in multiterms aggregations involving empty buckets (#7318)
• Precise system clock time in MasterService debug logs (#7902)
• Improve indexing performance for flat_object type (#7855)
• Adds log4j configuration for telemetry LogSpanExporter (#8393)
• Fix painless casting bug, which crashes the OpenSearch process (#8315)
• Add missing validation/parsing of SearchBackpressureMode of SearchBackpressureSettings (#7541)