opensearch-project/OpenSearch 3.3.0

on GitHub

Version 3.3.0 Release Notes

Compatible with OpenSearch and OpenSearch Dashboards version 3.3.0

Added

• Expand fetch phase profiling to support inner hits and top hits aggregation phases (#18936)
• [Rule-based Auto-tagging] add the schema for security attributes (#19345)
• Add temporal routing processors for time-based document routing (#18920)
• Implement Query Rewriting Infrastructure (#19060)
• The dynamic mapping parameter supports false_allow_templates (#19065, #19097)
• [Rule-based Auto-tagging] restructure the in-memory trie to store values as a set (#19344)
• Add a toBuilder method in EngineConfig to support easy modification of configs (#19054)
• Add StoreFactory plugin interface for custom Store implementations (#19091)
• Use S3CrtClient for higher throughput while uploading files to S3 (#18800)
• [Rule-based Auto-tagging] bug fix on Update Rule API with multiple attributes (#19497)
• Add a dynamic setting to change skip_cache_factor and min_frequency for querycache (#18351)
• Add overload constructor for Translog to accept Channel Factory as a parameter (#18918)
• Addition of fileCache activeUsage guard rails to DiskThresholdMonitor (#19071)
• Add subdirectory-aware store module with recovery support (#19132)
• [Rule-based Auto-tagging] Modify get rule api to suit nested attributes (#19429)
• [Rule-based Auto-tagging] Add autotagging label resolving logic for multiple attributes (#19486)
• Field collapsing supports search_after (#19261)
• Add a dynamic cluster setting to control the enablement of the merged segment warmer (#18929)
• Publish transport-grpc-spi exposing QueryBuilderProtoConverter and QueryBuilderProtoConverterRegistry (#18949)
• Support system generated search pipeline. (#19128)
• Add epoch_micros date format (#14669)
• Grok processor supports capturing multiple values for same field name (#18799)
• Add support for search tie-breaking by _shard_doc (#18924)
• Upgrade opensearch-protobufs dependency to 0.13.0 and update transport-grpc module compatibility (#19007)
• Add new extensible method to DocRequest to specify type (#19313)
• [Rule based auto-tagging] Add Rule based auto-tagging IT (#18550)
• Add all-active ingestion as docrep equivalent in pull-based ingestion (#19316)
• Adding logic for histogram aggregation using skiplist (#19130)
• Add skip_list param for date, scaled float and token count fields (#19142)
• Enable skip_list for @timestamp field or index sort field by default (#19480)
• Implement GRPC MatchPhrase, MultiMatch queries (#19449)
• Optimize gRPC transport thread management for improved throughput (#19278)
• Implement GRPC Boolean query and inject registry for all internal query converters (#19391)
• Added precomputation for rare terms aggregation (#18978)
• Implement GRPC Script query (#19455)
• [Search Stats] Add search & star-tree search query failure count metrics (#19210)
• [Star-tree] Support for multi-terms aggregation (#18398)
• Add stream search enabled cluster setting and auto fallback logic (#19506)
• Implement GRPC Exists, Regexp, and Wildcard queries (#19392)
• Implement GRPC GeoBoundingBox, GeoDistance queries (#19451)
• Implement GRPC Ids, Range, and Terms Set queries (#19448)
• Implement GRPC Nested query (#19453)
• Add sub aggregation support for histogram aggregation using skiplist (19438)
• Optimization in String Terms Aggregation query for Large Bucket Counts (#18732)
• New cluster setting search.query.max_query_string_length (#19491)
• Add StreamNumericTermsAggregator to allow numeric term aggregation streaming (#19335)
• Query planning to determine flush mode for streaming aggregations (#19488)
• Harden the circuit breaker and failure handle logic in query result consumer (#19396)
• Add streaming cardinality aggregator (#19484)
• Disable request cache for streaming aggregation queries (#19520)
• [WLM] add a check to stop workload group deletion having rules (#19502)

Changed

• Refactor if-else chains to use Java 17 pattern matching switch expressions (#18965)
• Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl (#18998)
• IllegalArgumentException when scroll ID references a node not found in Cluster (#19031)
• Adding ScriptedAvg class to painless spi to allowlist usage from plugins (#19006)
• Make field data cache size setting dynamic and add a default limit (#19152)
• Replace centos:8 with almalinux:8 since centos docker images are deprecated (#19154)
• Add CompletionStage variants to IndicesAdminClient as an alternative to ActionListener (#19161)
• Remove cap on Java version used by forbidden APIs (#19163)
• Omit maxScoreCollector for field collapsing when sort by score descending (#19181)
• Disable pruning for doc_values for the wildcard field mapper (#18568)
• Make all methods in Engine.Result public (#19276)
• Create and attach interclusterTest and yamlRestTest code coverage reports to gradle check task (#19165)
• Optimized date histogram aggregations by preventing unnecessary object allocations in date rounding utils (19088)
• Optimize source conversion in gRPC search hits using zero-copy BytesRef (#19280)
• Allow plugins to copy folders into their config dir during installation (#19343)
• Add failureaccess as runtime dependency to transport-grpc module (#19339)
• Migrate usages of deprecated Operations#union from Lucene (#19397)
• Delegate primitive write methods with ByteSizeCachingDirectory wrapped IndexOutput (#19432)
• Bump opensearch-protobufs dependency to 0.18.0 and update transport-grpc module compatibility (#19447)
• Bump opensearch-protobufs dependency to 0.19.0 (#19453)
• Disable query rewriting framework as a default behaviour (#19592)

Fixed

• Fix unnecessary refreshes on update preparation failures (#15261)
• Fix NullPointerException in segment replicator (#18997)
• Ensure that plugins that utilize dumpCoverage can write to jacoco.dir when tests.security.manager is enabled (#18983)
• Fix OOM due to large number of shard result buffering (#19066)
• Fix flaky tests in CloseIndexIT by addressing cluster state synchronization issues (#18878)
• [Tiered Caching] Handle query execution exception (#19000)
• Grant access to testclusters dir for tests (#19085)
• Fix assertion error when collapsing search results with concurrent segment search enabled (#19053)
• Fix skip_unavailable setting changing to default during node drop issue (#18766)
• Fix issue with s3-compatible repositories due to missing checksum trailing headers (#19220)
• Add reference count control in NRTReplicationEngine#acquireLastIndexCommit (#19214)
• Fix pull-based ingestion pause state initialization during replica promotion (#19212)
• Fix QueryPhaseResultConsumer incomplete callback loops (#19231)
• Fix the scaled_float precision issue (#19188)
• Fix Using an excessively large reindex slice can lead to a JVM OutOfMemoryError on coordinator (#18964)
• Add alias write index policy to control writeIndex during restore (#1511)
• [Flaky Test] Fix flaky test in SecureReactorNetty4HttpServerTransportTests with reproducible seed (#19327)
• Remove unnecessary looping in field data cache clear (#19116)
• [Flaky Test] Fix flaky test IngestFromKinesisIT.testAllActiveIngestion (#19380)
• Fix lag metric for pull-based ingestion when streaming source is empty (#19393)
• Fix IntervalQuery flaky test (#19332)
• Fix ingestion state xcontent serialization in IndexMetadata and fail fast on mapping errors (#19320)
• Fix updated keyword field params leading to stale responses from request cache (#19385)
• Fix cardinality agg pruning optimization by self collecting (#19473)
• Implement SslHandler retrieval logic for transport-reactor-netty4 plugin (#19458)
• Cache serialised cluster state based on cluster state version and node version (#19307)
• Fix stats API in store-subdirectory module's SubdirectoryAwareStore (#19470)
• Setting number of sharedArenaMaxPermits to 1 (#19503)
• Handle negative search request nodes stats (#19340)
• Remove unnecessary iteration per-shard in request cache cleanup (#19263)
• Fix derived field rewrite to handle range queries (#19496)
• Fix incorrect rewriting of terms query with more than two consecutive whole numbers (#19587)

Dependencies

• Bump com.gradleup.shadow:shadow-gradle-plugin from 8.3.5 to 8.3.9 (#19400)
• Bump com.netflix.nebula.ospackage-base from 12.0.0 to 12.1.1 (#19019, #19460)
• Bump actions/checkout from 4 to 5 (#19023)
• Bump commons-cli:commons-cli from 1.9.0 to 1.10.0 (#19021)
• Bump org.jline:jline from 3.30.4 to 3.30.5 (#19013)
• Bump com.github.spotbugs:spotbugs-annotations from 4.9.3 to 4.9.6 (#19015, #19294, #19358, #19459)
• Bump com.azure:azure-storage-common from 12.29.1 to 12.30.2 (#19016, #19145)
• Update OpenTelemetry to 1.53.0 and OpenTelemetry SemConv to 1.34.0 (#19068)
• Bump 1password/load-secrets-action from 2 to 3 (#19100)
• Bump com.nimbusds:nimbus-jose-jwt from 10.3 to 10.5 (#19099, #19101, #19254, #19362)
• Bump netty from 4.1.121.Final to 4.1.125.Final (#19103, #19269)
• Bump Google Cloud Storage SDK from 1.113.1 to 2.55.0 (#18922)
• Bump com.google.auth:google-auth-library-oauth2-http from 1.37.1 to 1.38.0 (#19144)
• Bump com.squareup.okio:okio from 3.15.0 to 3.16.0 (#19146)
• Bump Slf4j from 1.7.36 to 2.0.17 (#19136)
• Bump org.apache.tika from 2.9.2 to 3.2.2 (#19125)
• Bump org.apache.commons:commons-compress from 1.26.1 to 1.28.0 (#19125)
• Bump io.projectreactor.netty:reactor_netty from 1.2.5 to 1.2.9 (#19222)
• Bump org.bouncycastle:bouncycastle_jce from 2.0.0 to 2.1.1 (#19222)
• Bump org.bouncycastle:bouncycastle_tls from 2.0.20 to 2.1.20 (#19222)
• Bump org.bouncycastle:bouncycastle_pkix from 2.0.8 to 2.1.9 (#19222)
• Bump org.bouncycastle:bouncycastle_pg from 2.0.11 to 2.1.11 (#19222)
• Bump org.bouncycastle:bouncycastle_util from 2.0.3 to 2.1.4 (#19222)
• Bump com.azure:azure-core from 1.55.5 to 1.56.0 (#19206)
• Bump com.google.cloud:google-cloud-core from 2.59.0 to 2.60.0 (#19208)
• Bump org.jsoup:jsoup from 1.20.1 to 1.21.2 (#19207)
• Bump org.apache.hadoop:hadoop-minicluster from 3.4.1 to 3.4.2 (#19203)
• Bump com.maxmind.geoip2:geoip2 from 4.3.1 to 4.4.0 (#19205)
• Replace commons-lang:commons-lang with org.apache.commons:commons-lang3 (#19229)
• Bump org.jboss.xnio:xnio-nio from 3.8.16.Final to 3.8.17.Final (#19252)
• Bump actions/setup-java from 4 to 5 (#19143)
• Bump com.google.code.gson:gson from 2.13.1 to 2.13.2 (#19290, #19293)
• Bump actions/stale from 9 to 10 (#19292)
• Bump com.nimbusds:oauth2-oidc-sdk from 11.25 to 11.29.1 (#19291, #19462)
• Bump Apache Lucene from 10.2.2 to 10.3.0 (#19296)
• Add com.google.code.gson:gson to the gradle version catalog (#19328)
• Bump org.apache.logging.log4j:log4j-core from 2.25.1 to 2.25.2 (#19360)
• Bump aws-actions/configure-aws-credentials from 4 to 5 (#19363)
• Bump com.azure:azure-identity from 1.14.2 to 1.18.0 (#19361)
• Bump net.bytebuddy:byte-buddy from 1.17.5 to 1.17.7 (#19371)
• Bump lycheeverse/lychee-action from 2.4.1 to 2.6.1 (#19463)
• Exclude commons-lang and org.jsonschema2pojo from hadoop-miniclusters (#19538)
• Bump io.grpc deps from 1.68.2 to 1.75.0 (#19495)

Removed

• Enable backward compatibility tests on Mac (#18983)