hydrusnetwork/hydrus v625

Version 625

on GitHub

more ratings work and some QoL

• thanks to a user, we have several new items--
• in services->manage services, all numerical rating services have a new 'icon padding' value, which allows you to squeeze the stars together or push them apart. set to 0 to restore the old 'loading bar' way of presenting (square) numerical ratings
• the manage ratings dialog has better layout
• also, two new checkboxes in options->gui pages allow you to have the client ask you to auto-name new 'page of pages' on either creation or 'send pages to a new page of pages' events

options

• file->shortcuts is gone! the same panel is now under file->options->shortcuts. I still hate how this guy works and want to merge all the sub-panels into one big user-friendly list with a filter and stuff
• the 'custom shortcuts' panel is now viewable in non-advanced-mode
• options->sort/collect is now split into tag sort and file sort/collect. the 'namespace grouping sort' is now in the correct location under the tag box whoops
• options->media viewer has all hover/background related options sent to the new options->media viewer hovers
• options->gui pages has had a reshuffle and the session stuff is pulled out to a new options->gui sessions
• all the stuff in options->speed and memory now starts collapsed
• the options dialog should be a little shorter by default now
• rewangled how the 'replace left/right with primary/secondary' and 'numpad non-number keys are normal' options in shortcuts work in the sub-dialogs--they still appear to apply instantly in the dialogs, but they now reset correctly if you cancel out of the main dialog
• the edit shortcuts panel now only writes to disk on ok if you actually edited something

file relationships

• added a new 'media' file relationships shortcut command, file relationships: delete false positives within selection, and an equivalent menu command when you have multiple thumbs selected, delete all false-positive relationships these files' alternate groups have between each other, which removes false positives only between pairs of files in the selection, rather than the other 'remove false positives' command, which removes all known false positives. if you accidentally set a false positive on a pair, this is now specifically undoable by selecting the pair and hitting this
• when you do the various 'delete false positives' actions, you now get a popup saying how many were deleted
• gave the 'file relationships' file menu a quick pass. it stacks better and now says when a file has no relationships. the reset/remove submenus are also split apart to clarify action importance. this horrible thing should nonetheless absolutely be replaced, wholesale, with a nice review and/or edit panel
• cleaned up some of the various duplicate merge logic, replacing hacky stuff with unified calls etc.. I think some redundant potentials are now removed more smartly when you merge certain alternate group members together, and I think the 'confirmed alternates' list is properly cleared of B entries when you merge B into A now. some 'merge alternate groups when two files are set dupe' stuff is clearer and works more thoroughly too. I expect to add a 'clear orphans' maintenance command here in future to handle these legacy issues and cleverly fix any db damage
• when you explicitly set files as alternate, any previous false positive relationship between the two is now repealed rather than the job silently failing

duplicates

• the 'A and B are visual duplicates' test now gives a negative result if either file has transparency, confidently if only one does and not if both do--it isn't smart enough to handle this yet
• the approve/deny buttons in semi-automatic auto-resolution rule review now do their work off the main thread and no longer block the UI on big jobs
• the 'preview' panel of an auto-resolution rule now fetches each block of pairs in a random order it figures out every dialog boot. if you open up a rule that has worked a bunch already, you can sit there watching it climb up to 500,000/700,000 and find nothing since it already did that front work. now it fetches randomly and will start population more swiftly. of course the results will now be in a roughly different order every time you boot the dialog, so that's not amazing--let me know how you find it. I think I may be adding a lot more random access like this, and that means where we will want unified pair sorting settings (much like we want in the manual duplicate filter, which still produces a garbage hardcoded fixed order of pairs!)

subs and downloads

• a core multi-column list update routine is optimised significantly. when many new items come to a list after an edit event (usually mass add or some mass edits), it now selects all items at once. previously it was doing so one at a time, which was triggering a billion button update calls. doing a mass paste into a subscription, or doing a 'separate sub of 1000 queries into 1000 separate subs of one query' now takes about three seconds, instead of many minutes
• when you paste queries into a subscription, if some already exist in the sub but are DEAD, you are now yes/no asked if you want to revive them
• if a subscription has a gallery page that decides searching should stop, which for this purpose typically means it has 'caught up' to a previous sync, all additional pending gallery url jobs of the same url class in the gallery log will now be set to 'skipped' with a reason of 'previous blah url said: (original stop reason)'. this ensures that downloaders that produce multiple gallery pages in one step are now processed efficiently and obey the catch-up mechanic rather than overspilling the cull limit and adding old files back in--as soon as one page is 'caught up', the rest are skipped. I made this only skip of the same url class so as not to break NGUG subscriptions. if an NGUG produces multiple gallery pages of the same class, they may be thrown off by this since they will no longer check every outstanding gallery url unless the previous produced something useful--if you are clever enough to know what this means, let me know how you get on
• when a network job downloads a file from a Post URL, it now registers the bandwidth use to the Post URL's domains as well, if they differ. this should soon fix the issue where a site that produces files on completely different domain was not pausing when the file domain hits its bandwidth rules (file urls override bandwidth rules within three seconds in order to keep post/file url hits close together, which helps some token stuff, but if the post url never becomes aware of how much bandwidth it is eating, it just keeps on chugging. now it won't!)
• added a checkbox to stop the 'override bandwidth in 3 seconds' rule under options->downloading, but I'm hoping the above overall fixes the issue

misc

• the page tab menu gets a new 'collapse pages' submenu. depending on where you click, it'll say 'this page', 'pages from here to the right', and 'pages to the right'. it will suck up all the files from the given pages and place them in a new locked search page, closing the old pages in the same step. use this for when you have eight small import pages in a row you want to collapse into a single processing page. it isn wrapped in a yes/no. it works on 'page of pages' too. this action is cool to do, so try it out
• the 'send down to a new page of pages' gets the same 'pages from here to the right' option too, so you can do both operations from the very left-hand side if that's what you want
• fixed a benign error popup when you hold the 'next image' shortcut while closing out on the last pair in a duplicates filter
• if attempting to send a file to the OS recycle bin results in a 0x80270021 error (many causes for this, but it seems it can also be an unluckily-timed sharing violation or a recycle bin with 100k+ items in it and being mega laggy), it is now re-attempted twice, with short breaks in between
• when, under services->manage services, you open an edit panel for a rating service that has stars, the client now repopulates the cache it stores of all your current custom svgs. no need to reboot now, just reload the dialog and it'll show what is current
• I may have solved some indirect threading deadlocks. all threaded jobs also initialise just a tiny bit faster now
• blurhash generation, which can be a whack of CPU per import, is now a tiny bit more efficient

boring stuff

• removed setuptools from the requirements.txts that don't have pyinstaller. we freeze the setuptools version to ensure a stable frozen build structure, so this isn't explicitly needed anywhere else
• refactored some of my advanced dev requirements.txts
• brushed up the 'running from source' help doc and added some info about testing/installing pip and venv, which aren't always available by default in Linux
• replaced all the typing.Collection style typedefs across the program with the collections.abc equivalents. this stuff was deprecated starting Python 3.9 lol. also, I'm pretty sure this was already true, but hydrus will now definitely not run in Python 3.8
• replaced all the basic typing.List style typedefs across the program with the standard list equivalents. same deal as above
• overhauled the 'db duplicates' unit tests, breaking up the previous monolith rollercoaster into something a bit more atomic. it is still a huge mess, but much better than before
• added some unit tests for the new 'remove false positives within selection'