hydrusnetwork/hydrus v621

Version 621

on GitHub

more ratings

• thanks to a user, we have many new rating options--
• options->media viewer and options->thumbnails now let you alter the size of ratings!
• there are now many new rating shapes to choose from under services->manage services: triangles pointing up/down/left/right; diamond; rhombus pointing right/left; pentagon; hexagon and small hexagon; six point star; eight point starburst; hourglass; large + and X shapes
• also, in an experiment, you can now also set a custom svg to your rating. there's a new folder, install_dir/static/star_shapes, that you can drop svgs into to have them available. I've put a couple of simple examples in already. try to do a squarish shape with a clear sillhouette on transparency. we will keep working on this, but feel free to try it out yourself and give feedback. I suspect we'll migrate all the existing custom polygons to nicer svgs in future if and when we nail nice svg border colours and stuff
• I added a help button to the manage services edit panel here, linking to the ratings help, which has a new section on the svg rating shapes
• I reworked some stuff behind the scenes to more neatly integrate this rating tech and allow for svg ratings. the 'edit service star shape' UI in manage services has two dropdowns now, too
• we are experimenting with fractional pixel sizing and drawing here, so some of the new ratings may seem a little blurry in places and/or not line up well, including between the underlying canvas and the top-right hover pop-up. I'm going to work on this a bit and see what I can do to make it more pixel-perfect
• we may figure out unicode character ratings in future also

misc

• fixed a bug in the duplicates system's 'pixel hash' test. my current pixel hash is dimensionless, and thus, by accident, a completely flat-colour (or careful checkerboard etc..) image of 100x500 has the same pixel hash as the same colour of 200x250. it obviously doesn't come up much, but I regret the error. rather than recompute everyone's pixel hashes with dimension data, we are first patching the test code to double-check the images both have the same width. it seems ok here, but let me know if your 'must (not) be pixel duplicates' duplicate file searches are now interminably slow
• split the prefetch settings in options->speed and memory into their own box, and updated the labels a bit
• added a setting to this new box to govern how many pairs the duplicate filter will prefetch. previously this was hardcoded to 1, now it defaults to 5 and you can set whatever
• when the client or server checks if it is already running, the 'client_running' file's 'process creation time' is now only tested to within a second's accuracy. if you have a bespoke backup workflow that involves creating your own client_running file, it should get caught better now despite float imprecision
• updated the various Linux install/running-from-source stuff to talk about libmpv2 as well as libmpv1.
• I rolled out a v620a hotfix last week after my refactoring accidentally broke the manual duplicate filter. sorry for the trouble! I have improved my testing regime to catch this in future

duplicates auto-resolution

• you can now add 'system:known url' and 'system:number of urls' predicates to the 'test A or B' comparators
• you can now add 'system:number of urls' to the 'test A against B using file info' comparators
• added unit tests for these, and cleaned up some of the unit tests in this area

avif

• the pillow-avif-plugin test last week seemed to go well. this library is now folded into the main builds, and AVIF rendering should be more reliable for the time being (issue #1720)
• if you run from source, you might like to rebuild your venv this week to get this

psd_tools replacement

• psd-tools is no longer used by the program
• thanks to a user report, we discovered that the Linux build was including an old and vulnerable version of a bz2 decoder. this thing was hanging around because of a drawing library included by a psd parser we use. I am not sure if the decoder was ever called, given what we use the library for. I did a test build of Linux with the bad file simply deleted, and it seemed to boot ok, but I wasn't sure if the file was bundled into a pyd on the Windows and macOS side, and I'm pretty sure it would be included in a source install, so I wasn't really happy
• ultimately I decided I didn't like having this complicated psd library hanging around when we only needed to pull dimensions, icc profile existence, and pulling an embedded render preview file, so I cobbled together some other backup solutions we already mostly figured out, including an ffmpeg renderer, and wrote a simple file parser and replaced all the psd_tools stuff with our own calls
• there are some psd files ffmpeg seems not to render as good as psd_tools did, but I'm fine with it for now
• I seem to be detecting icc profiles that psd tools did not recognise, so I'm scheduling all psds for a 'has an icc profile?' check on update. maybe I'm false-positiving, but I'll make sure the existing store reflects current code at least
• I also fixed some psd rendering for files that had weird 100% transparency
• along the way I improved some PSD error messages and handling. PSDs that have no embedded preview should stop spamming that info to the log on thumbnail regen

swfrender removal

• since cleaning was on my mind, I'm removing the swfrender exes too. new swf files you import will just get default 'flash' thumbnails from now on. having these ancient executables hanging around to load unknown flash files is not an excellent idea these days, even though it has been fun
• the actively developed 'Ruffle' flash emulator seems to be working on a CLI executable that can render flash frames, which could be a much better and more reliable replacement in future
• I haven't played with it properly yet, but ffmpeg can apparently thumbnail some flv-embed-style flash files, so I'll explore this too

refactoring and cleanup

• moved some ffmpeg stuff from HydrusVideoHandling to the new HydrusFFMPEG
• merged all 'ffmpeg was missing' and 'ffmpeg output no content' error handling to single calls and improved coverage
• wrote some 'render to pipe' stuff for ffmpeg to make some of the ffmpeg based rendering work a little quicker and without the temp dir
• deleted the HydrusPSDTools file
• moved some more old network reporting mode code to a cleaner unified method