Happy Holidays from your favorite degeneracy library!
I am bringing you the long-awaited gift of indexes
to cap off the year.
๐ Important changes
If you don't want to read the rest of the changelog:
- If you're running from source, LRR now requires two supplementary Redis databases, using indexes 2 and 3 by default. You can change those numbers in
lrr.conf
. (Onlydatabase_search
is used for now, config will come into use next release) - If you update a large instance, search will not work for a bit while the indexes are built -- You can follow the status of the
stat_indexes
Minion job to see when you can start searching again. - The
/api/archives/random
endpoint has slightly changed, which means third-party clients might have broken random functionality until you update. - OPDS-PSE support is in, but won't work for archives that don't use JPEG for their images.
๐ Search has been rewritten (#555, #444, #563, #657 and more!)
Search has always been the biggest performance bottleneck of the entire server due to using a rather dumb algorithm that always did multiple database passes...
Through the incredible invention of search indexes but for redis, I'm happy to announce that search is now up to 10 times faster for the most used scenarios.
088search_Trim.mp4
Some notes on the search rewrite:
- Spaces are no longer a delimiter for different tags in a search prompt, commas should now be used instead. This matches what the autocompletion box does and how metadata as a whole is entered into the app, so I don't think this will cause too much pain.
- Search now requires the index tables to be built (does so on app start, and then updates them as tags/titles/etc are modified), so large instances might take a few mins to get a usable search again after updating.
- Title searches will work the same as before (quotes or $ for an exact match, ? * _ % as wildcards. )
- For tags, if you don't use exact matches,
tag
searches will be wildcarded on both ends (like previous behavior),namespace:tag
searches will only be wildcarded at the end (namespace stays intact).
You can use *namespace:tag if you want the old behavior for some weird reason.
This was one of the biggest jobs on the list, and while it took a few days to get it right I hope you'll enjoy it -- I do think it's quite a gamechanger.
This rewrite doesn't optimize everything (large intersections and sorting by anything else than title will still be a bit slow), so there's certainly still room for improvement.
The new system makes heavy use of Redis sets and sorted sets, which are all lodged in a separate, third database.
See the DB Architecture document. (the config keys moving to a fourth database is a pending change that will land in a future release.)
๐ OPDS PSE Compliance
I've dusted off the OPDS API to make it use the new Search, and took the opportunity to implement a few niceties.
LANraragi now supports OPDS PSE 1.1, which means you can use compatible OPDS clients to read archives page-by-page from the server and update your reading progression. (If server-side progress tracking is enabled)
As a known caveat, OPDS-PSE will currently not work if your archive contains non-JPEG image files.
Considering the surprisingly complete state of third-party LRR clients, I don't think many people use OPDS at all, but it's nice to be a good citizen to standards nonetheless.
And if you can't sideload DuReader, that means you can now use Panels on iOS! At least until Apple finally caves in and allows sideloading on their devices. god bless the EU
Other changes
-
(#646) New EH layout compatibility
๐ผ Both ex and e-h have recently updated their layout and search API - While the old plugin still works fine on the new layouts, it still got a small update to benefit from the changes.
๐ You can now search by gID -- this will fall back to title name if no gId found or search failed. (Thanks @polak14 !)
๐ช Theigneous
cookie is now supported, which fixes the problem of using ex in non European and American regions. (Thanks @zizzdog !) -
(#709) Carousel now auto-reloads with every new search + has a nicer empty view
๐ whee
๐ I couldn't really afford to make the carousel auto-reload before since it relies on Search, which had...yeah. -
Add basic Redis password support
๐ You can now set
redis_password
inlrr.conf
in case your Redis server is password-protected. -
Windows fixes
๐ช The WSL detection should now work regardless of your system language.
๐ ๏ธ The Windows app should now detect "file not found" WSL failures better.
๐ง The repair button is also now always available from Settings. -
Miscellaneous bugfixes and improvements
๐ Login now redirects you to the page you tried to access beforehand
๐ Update FontAwesome to v6
๐ Update Mojolicious to v9.30
๐ (#396) Allow most common http methods when CORS is enabled
๐ Fix not being able to redownload an URL after having deleted it from the server without a restart -
API Changes
๐
/api/search/random
now uses the same object syntax as/api/search
(arcid instead of id, newonly instead of isnew)
๐?key=
can be used again to authenticate API requests. (Mostly just for OPDS, please don't actually use this)
Full Changelog: v.0.8.7...v.0.8.8
Inspired by his and Enoโs trip to the Gugging Psychiatric Clinic in 1994, [...] โDerangedโ seems mainly Enoโs work, though one ancestor was Bowie and Nile Rodgersโ โReal Cool Worldโ (thereโs also an echo of โBillie Jeanโ in its opening four-note synth hook)