Major Improvement
We optimize the external centroid index building speed, about 30%. Now it takes about 30h to build index for 100M vectors with only 4 vcpu on i4i.xlarge.
What's Changed
- refactor: move algorithm to a crate by @usamoi in #172
- feat: pinning index in memory when building, second try by @usamoi in #181
- fix: use linked list of vectors to skip realloc by @usamoi in #182
- feat: use select algorithm to replace heap, if k in top-k is expected to be small by @usamoi in #183
- ci: install pg13 in docker image by @usamoi in #186
- ci: use less docker by @usamoi in #187
- feat: rerank by fetching vectors in heap table by @usamoi in #189
- ci: enable CI for pg13 by @usamoi in #185
- chore: update dependencies by @usamoi in #190
- refactor: remove meaningless target feature requirements by @usamoi in #192
- fix: test simd operations in emulator by @usamoi in #193
- feat: neon impl of u8::reduce_sum_of_x by @usamoi in #194
- chore: update 0.2.1 schema (upgrade) script by @usamoi in #195
Full Changelog: 0.2.0...0.2.1