hydrusnetwork/hydrus v646 on GitHub

misc

I made mpv safer, both in the existing recycle system and the create/destruction test. if you tried the mpv test last week and got hangs when flicking quickly or when leaving certain media viewers on an mpv window, please give it another go
when pages load themselves initially, the individual file load jobs are split into different work packets for the worker pool, so a handful of big pages will no longer monopolise the queue. also, if a page is closed, the initial load pauses--if it is undo-reopened, initial load resumes
in the duplicate filter, when the difference in import time is less than 30 days, the 'imported at similar time (unhelpful average timestamp)' label is replaced with 'a little newer/older than' (issue #1898)
if you have a very large database, it now requires up to 5GB of free disk space on the db partition to boot (the cap was previously 512MB)
the db disk space check now occurs on shutdown too. if you have less space than it thinks is safe, it warns you that shutdown may not save correctlly and you should immediately free up some space. you have the choice of backing out or going ahead (issue #1895)

when potential duplicate pairs are counted or searched with just one small creator tag or a system:hash or something, and the final result is tiny, like 5 out of 750,000, it now won't iterate through your whole pair store but instead do a few blocks and then immediately come up with the answer in one step (issue #1778)
this works by examining the running sample, and if we are confident the hit-rate is lower than 1%, the search strategy now inverts, and rather than iterating through 750,000 pairs to find 5 that match the search context, it runs the underlying (typically very small and fast) file search and runs those n files against the 750,000 rows, getting the 5 hits
it should all just work, but let me know how it goes. does it kick in too late, too infrequently? are there search types it lags out at?
I've added a 'potential duplicates report mode' to help->debug->report modes that spits a bunch of search data to log. if you are into all this, please run it on a variety of searches in IRL situations and copy/paste to me
this was the last difficult job for duplicates auto-resolution. I've now got about a dozen small jobs for comparator tweaks and stuff, and maybe some smarter count update tech so we aren't resetting search spaces so much, and then this system is v1.0 done. I still feel good about hitting this by the end of the year

the routine that performs the 'search duplicate pairs in small increments' iteration now has a unified object to govern the search. it handles search space initialisation/reset, search progress, reset, block-popping, block-throttling, hitrate tracking, estimate confidence intervals, desired total hits, status reports, and now search strategy
put this new fragmentary search into the potential duplicate search context panel count call and the Client API version
put it in group/mixed filtering pairs fetch, the 'fetch some random', and the Client API version
put it in the auto-resolution preview panel thumbnail pair fetch
added another wilson interval confidence test to the fragmentary test to do 'are we 95% sure the hitrate is below x%?'
added some logic to figure out if a one-time file search or the remaining iterative search is going to be faster, including if the caller only wants n hits, and I profiled stuff a bit so I could establish a magic coefficient
the search space randomisation strategy is now based on whether the searcher is looking to stop/switch early or always wants to do the whole job
deleted some old pair-fetch code that is no longer used by the Client API since the pairfactory overhaul
updated my Client API unit tests for potential pair fetch to use nicer db mocking to handle some cleverer fragmentary update stuff properly
wrote some neater db routines for navigating these questions
cleaned up some search optimisations in here. not that significant, just edge cases

all media viewers will now defer any media transition if they are currently looking at an mpv window that is still initialising. once the mpv window is ok, they'll recall the most recent set-media request and move on. this seems to fix the 'spawn errors/hang the client when scrolling through many mpv windows fast' issue in the mpv destruction test, and some related jank
all media viewer top level windows (i.e. not the preview) now immediately ignore window close events if the current mpv player is not yet initialised
in the new mpv destruction test, mpv windows are put in a holding queue and the mpv handle explicitly terminated before Qt widget deletion. previously this was handled by the python garbage collector, which is not ideal
when any top level media window (i.e. not the previews) gets a close signal, if there are any mpv windows awaiting destruction, the window hides itself and waits until mpv is clear before allowing Qt to destroy it
in both the 'is initialised?' and 'is cleaned up?' checks, we just go ahead if it has been 180 seconds
fixed an mpv options-setting bug that could sometimes print an error to log on shutdown

I think I fixed an issue where some thread jobs could not terminate correctly if the UI window they were attached to died before the job was done. this may be related to some hanging clients that have extremely busy sessions
all multi-column temp integer tables in the db are now row-unique
fixed an issue where a couple of shutdown-late CallAfter guys could try to do a CallAfter after Qt was down and the log was closed out, which would spam some error to the terminal
cleaned up some media viewer close logic
the thumbnail-preview focus-media logic is now more cleverly idempotent and stops spamming some excess update signals
all async updaters have names that now render nice in the 'review threads' debug panel (we're chasing down a guy that seems to be stuck on one client)