weaviate/weaviate v1.8.0-rc.3

[Pre-release] v1.8.0-rc.3 - Pagination

on GitHub

New features compared to the previous rc version:

• Pagination #1627

  Starting with this release search results can now be paged. This feature is available on:

  • List requests (GET /v1/objects and GraphQL Get { Class { } })
  • Vector Searches (GraphQL near<Media>) and Filter Searches (GraphQL where: {})

  Usage

  To use pagination, one new parameter is introduced (offset) which works in conjunction with the existing limit parameter. For example, to list the first ten results, set limit: 10. Then, to "display the second page of 10", set offset: 10, limit:10 and so on. E.g. to show the 9th page of 10 results, set offset:80, limit:10 to effectively display results 81-90.

  To do so in REST, simply append the two parameters as URL params, e.g. GET /v1/objects?limit=25&offset=75
  To do so in GraphQL, simply add the two parameters to the class, e.g. { Get { MyClassName(limit:25, offset: 75) { ... } } }

  Performance and Resource Considerations & Limitations

  The pagination implementation is an offset-based implementation, not a cursor-based implementation. This has the following implications:

  • The cost of retrieving one further page is higher than that of the last. Effectively when searching for search results 91-100, Weaviate will internally retrieve 100 search results and discard results 0-90 before serving them to the user. This effect is amplified if running in a multi-shard setup, where each shard would retrieve 100 results, then the results aggregated and ultimately cut off. So in a 10-shard setup asking for results 91-100 Weavaite will effectively have to retrieve 1000 results (100 per shard) and discard 990 of them before serving. This means, high page numbers lead to longer response times and more load on the machine/cluster.
  • Due to the increasing cost of each page outlined above, there is a limit to how many objects can be retrieved using pagination. By default setting the sum of offset and limit to higher than 10,000 objects, will lead to an error. If you must retrieve more than 10,000 objects, you can increase this limit by setting the environment variable QUERY_MAXIMUM_RESULTS=<desired-value>. Warning: Setting this to arbitrarily high values can make the memory consumption of a single query explode and single queries can slow down the entire cluster. We recommend setting this value to the lowest possible value that does not interfere with your users' expectations.
  • The pagination setup is not stateful. If the database state has changed between retrieving two pages there is no guarantee that your pages cover all results. If no writes happened, then pagination can be used to retrieve all possible within the maximum limit. This means asking for a single page of 10,000 objects will lead to the same results overall as asking for 100 pages of 100 results.

• Filtered Vector Search Flat Search Cutoff #1728 #1729

  As outlined in this article, you can now configure a switch to a flat search when a filtered HNSW search would become too expensive due to the restrictiveness of the filter.