Version 7.0.0

Released: Jan 30th 2025

➡️➡️➡️ DOWNLOAD HERE ⬅️⬅️⬅️

📢📢📢Announcement blog post📢📢📢

Major changes

• Issue #1497 Added new Fuzzy Search and Autocomplete functionality for easier searching.
• Issue #1500 Integration with Kafka.
• Issue #1928 Introduced secondary indexes for JSON.
• Issue #2361 Updates and searches during updates are no longer blocked by chunk merging.
• Issue #2787 Automatic disk chunk flush for RT table to improve performance; now, we automatically flush a RAM chunk to a disk chunk, preventing performance issues caused by the lack of optimizations in RAM chunks, which could sometimes lead to instability depending on chunk size.
• Issue #2811 Scroll option for easier pagination.
• Issue #931 Integration with Jieba for better Chinese tokenization.

Minor changes

• ⚠️ BREAKING Issue #1111 Fixed support for global_idf in RT tables. Requires table recreation.
• ⚠️ BREAKING Issue #2103 Removed Thai characters from internal cjk charset. Update your charset definitions accordingly: if you have cjk,non_cjk and Thai characters are important for you, change it to cjk,thai,non_cjk, or cont,non_cjk, where cont is the new designation for all continuous-script languages (i.e., cjk + thai). Modify existing tables using ALTER TABLE.
• ⚠️ BREAKING Issue #2458 Incorporated zlib support in Windows packages.
• ⚠️ BREAKING Issue #2468 CALL SUGGEST / QSUGGEST now compatible with distributed tables. This increases the master/agent protocol version. If you are running Manticore Search in a distributed environment with multiple instances, make sure to first upgrade the agents, then the masters.
• ⚠️ BREAKING Issue #2889 Changed column name from Name to Variable name for PQ SHOW META.
• ⚠️ BREAKING Issue #879 Introduced per-table binlog with new options: binlog_common, binlog for create table / alter table. You need to make a clean shutdown of the Manticore instance before upgrading to the new version.
• ⚠️ BREAKING Issue #1789 Fixed incorrect error message when a node joins a cluster with the wrong replication protocol version. This change updates the replication protocol. If you are running a replication cluster, you need to:
  • First, cleanly stop all your nodes
  • Then, start the node that was stopped last with --new-cluster, using the tool manticore_new_cluster in Linux.
  • Read about restarting a cluster for more details.
• ⚠️ BREAKING Issue #2308 Added support for multiple tables in ALTER CLUSTER ADD and DROP. This change also affects the replication protocol. Refer to the previous section for guidance on handling this update.
• Issue #2997 Fixed dlopen issue on Macos.
• Commit 4954 Changed the default cutoff for OPTIMIZE TABLE on tables with KNN indexes to improve search performance.
• Commit cfc8 Added COUNT(DISTINCT) support for ORDER BY in FACET and GROUP BY.
• Issue #1103 Improved clarity in logging chunk merging.
• Issue #1130 Added support for DBeaver.
• Issue #1546 Implemented secondary indexes for POLY2D()/GEOPOLY2D() functions.
• Issue #1630 HTTP requests now support Content-Encoding: gzip.
• Issue #1831 Added SHOW LOCKS command.
• Issue #2187 Allowed Buddy request to daemon to bypass the searchd.max_connections constraint.
• Issue #2208 Added support for joining tables via the JSON HTTP interface.
• Issue #2235 Log successfully processed queries via Buddy in their original form.
• Issue #2249 Added special mode to run mysqldump for replicated tables.
• Issue #2268 Improved external files renaming on copy for CREATE TABLE and ALTER TABLE statements.
• Issue #2402 Updated searchd.max_packet_size default to 128MB.
• Issue #2419 Added support for IDF boost modifier in JSON "match".
• Issue #2430 Enhanced binlog writing synchronization to prevent errors.
• Issue #2479 Added support for SHOW TABLE INDEXES command.
• Issue #2485 Set session metadata for Buddy replies.
• Issue #2490 Millisecond resolution for aggregations at compatibility endpoints.
• Issue #2500 Changed error messages for cluster operations when replication fails to start.
• Issue #2584 New performance metrics in SHOW STATUS: min/max/avg/95th/99th percentile per query type for the last 1, 5, and 15 minutes.
• Issue #2639 Replaced all instances of index with table in requests and responses.
• Issue #2643 Added distinct column to HTTP /sql endpoint aggregation results.
• Issue #268 Implemented autodetection of data types imported from Elasticsearch.
• Issue #2744 Added collation support for string JSON field comparison expressions.
• Issue #2752 Added support for uuid_short expression in the select list.
• Issue #2783 Manticore Search now runs Buddy directly without the manticore-buddy wrapper.
• Issue #2785 Differentiated error messages for missing tables and tables that do not support insert operations.
• Issue #2789 OpenSSL 3 is now statically built into searchd.
• Issue #2790 Added CALL uuid_short statement to generate sequences with multiple uuid_short values.
• Issue #2803 Added separate options for the right table in the JOIN operation.
• Issue #2810 Improved HTTP JSON aggregation performance to match GROUP BY in SphinxQL.
• Issue #2854 Added support for fixed_interval in Kibana date-related requests.
• Issue #2909 Implemented batching for JOIN queries, which improves the performance of certain JOIN queries by hundreds or even thousands of times.
• Issue #2937 Enabled the use of joined table weight in fullscan queries.
• Issue #2953 Fixed logging for join queries.
• Issue #337 Hid Buddy exceptions from searchd log in non-debug mode.
• Issue #2931 Daemon shutdown with error message if user sets wrong ports for replication listener.

Bug fixes

• Commit 0c6b Fixed: Incorrect results returned in JOIN queries with more than 32 columns.
• Issue #2335 Resolved an issue where table join failed when two json attributes were used in the condition.
• Issue #2338 Fixed incorrect total_relation in multiqueries with cutoff.
• Issue #2366 Corrected filtering by json.string in the right table on table join.
• Issue #2406 Enabled use of null for all values in any POST HTTP JSON endpoints (insert/replace/bulk). In this case a default value is used.
• Issue #2418 Optimized memory consumption by adjusting allocation of the max_packet_size network buffer for the initial socket probe.
• Issue #2420 Fixed insertion of unsigned int into bigint attribute via JSON interface.
• Issue #2422 Fixed secondary indexes to work correctly with exclude filters and pseudo_sharding enabled.
• Issue #2423 Addressed a bug in manticore_new_cluster.
• Issue #2448 Resolved daemon crash on malformed _update request.
• Issue #2452 Fixed histograms' inability to handle value filters with excludes.
• Issue #55 Fixed knn queries against distributed tables.
• Issue #68 Enhanced handling of exclude filters on table encoding in columnar accessor.
• Commit 0eb1 Fixed expression parser not obeying redefined thread_stack.
• Commit c304 Fixed a crash on cloning columnar IN expression.
• Commit edad Fixed the inversion issue in the bitmap iterator which caused a crash.
• Commit fc30 Fixed issue where some Manticore packages were automatically removed by unattended-upgrades.
• Issue #1019 Improved handling of queries from DbForge MySQL tool.
• Issue #1107 Fixed escaping of special characters in CREATE TABLE and ALTER TABLE. ❤️ Thank you, @subnix for the PR.
• Issue #116 Fixed a deadlock when updating a blob attribute in a frozen index. The deadlock occurred due to conflicting locks while trying to unfreeze the index. It could cause a failure in manticore-backup too.
• Issue #1818 OPTIMIZE now throws an error when the table is frozen.
• Issue #2001 Allowed function names to be used as column names.
• Issue #2153 Fixed daemon crash on querying table settings with unknown disk chunk.
• Issue #2184 Fixed an issue where searchd hangs on stop after FREEZE and FLUSH RAMCHUNK.
• Issue #2228 Removed date/time related tokens (and regex) from the reserved words.
• Issue #2255 Fixed crash on FACET with more than 5 sort fields.
• Issue #2265 Fixed failure to restore mysqldump with index_field_lengths enabled.
• Issue #2291 Fixed crash when executing ALTER TABLE command.
• Issue #2333 Fixed MySQL DLL in the Windows package for the indexer to work properly.
• Issue #2393 Fixed GCC compilation error. ❤️ Thank you, @animetosho for the PR.
• Issue #2447 Fixed an escaping issue in _update.
• Issue #2460 Fixed indexer crash when declaring multiple attributes or fields with the same name.
• Issue #2467 Resolved crash of daemon on incorrect transform for nested bool queries for the "compat" search related endpoints.
• Issue #2493 Fixed expansion in phrases with modifiers.
• Issue #2535 Resolved daemon crash on using ZONE or ZONESPAN fulltext operator.
• Issue #2552 Fixed infix generation for plain and RT tables with keywords dictionary.
• Issue #2571 Fixed error reply in FACET query; set default sort order to DESC for FACET with COUNT(*).
• Issue #2580 Fixed daemon crash on Windows during startup.
• Issue #2603 Fixed query truncation for HTTP endpoints /sql and /sql?mode=raw; made requests from these endpoints consistent without requiring query= header.
• Issue #2623 Fixed issue where auto-schema creates a table but fails simultaneously.
• Issue #2627 Fixed HNSW library to support loading multiple KNN indexes.
• Issue #2630 Fixed freeze issue when multiple conditions occur simultaneously.
• Issue #2645 Fixed crash with fatal error when using OR with KNN search.
• Issue #2647 Fixed issue where indextool --mergeidf *.idf --out global.idf deletes the output file after creation.
• Issue #2658 Fixed daemon crash on sub-select with ORDER BY string in outer select.
• Issue #2686 Fixed crash on updating float attribute alongside string attribute.
• Issue #2704 Fixed issue where multiple stopwords from lemmatize_xxx_all tokenizers increase hitpos of subsequent tokens.
• Issue #2708 Fixed crash on ALTER ... ADD COLUMN ... TEXT.
• Issue #2737 Fixed issue where updating a blob attribute in a frozen table with at least one RAM chunk causes subsequent SELECT queries to wait until the table is unfrozen.
• Issue #2742 Fixed query cache skip for queries with packed factors.
• Issue #2775 Manticore now reports an error on unknown action instead of crashing on _bulk requests.
• Issue #2791 Fixed return of inserted document ID for HTTP _bulk endpoint.
• Issue #2797 Fixed crash in grouper when handling multiple tables, one of which is empty and the other has a different number of matched entries.
• Issue #2835 Fixed crash in complex SELECT queries.
• Issue #2872 Added error message if ALL or ANY argument in IN expression is not a JSON attribute.
• Issue #2882 Fixed daemon crash on updating MVA in large tables.
• Issue #2888 Fixed crash on tokenization failure with libstemmer. ❤️ Thank you, @subnix for the PR.
• Issue #2919 Fixed an issue where the joined weight from the right table was not working correctly in expressions.
• Issue #2919 Fixed issue where right-joined table weight does not work in expressions.
• Issue #325 Fixed CREATE TABLE IF NOT EXISTS ... WITH DATA failure when the table already exists.
• Issue #351 Fixed undefined array key "id" error on count by KNN with document ID.
• Issue #359 Fixed REPLACE INTO cluster_name:table_name functionality.
• Issue #67 Fixed FATAL error when running Manticore Docker container with --network=host.