hydrusnetwork/hydrus v545

Version 545

on GitHub

blurhash

• thanks to a user's work, hydrus now calculates the blurhash of files with a thumbnail! (issue #394)
• if a file has no thumbnail but does have a blurhash (e.g. missing files, or files you previously deleted and are looking at in a clever view), it now presents a thumbnail generated from that blurhash
• all existing thumbnail-having files are scheduled for a blurhash calculation (this is a new job in the file maintenance system). if you have hundreds of thousands of files, expect it to take a couple of weeks/months to clear. if you need to hurry this along, the queue is under database->file maintenance
• any time a file's thumbnail changes, the blurhash is scheduled for a regen
• for this first version, the blurhash is very small and simple, either 15 or 16 cells for ~34 bytes. if we end up using it a lot somewhere, I'd be open to making a size setting so you can set 8x8 or higher grids for actually decent blur-thumbs
• a new help->debug report mode switches to blurhashes instead of normal thumbs

file history search

• I did to the file history chart (help->view file history) what I did to mr bones a couple weeks ago. you can now search your history of imports, archives, and deletes for creator x, filetype y, or any other search you can think of
• I hacked this all together right at the end of my week, so please bear with me if there are bugs or dumb permitted domains/results. the default action when you first open it up should all work the same way as before, no worries™, but let me know how you get on and I'll fix it!
• there's more to do here. we'll want a hideable search panel, a widget to control the resolution of the chart (currently fixed at 7680 to look good blown up on a 4k), and it'd be nice to have a selectable date range
• in the longer term future, it'd be nice to have more lines of data and that chart tech you see on financial sites where it shows you the current value where your mouse is

client api

• the file_metadata call now says the new blurhash. if you pipe it into a blurhash library and blow it up to an appopriate ratio canvas, it should just work. the typical use is as a placeholder while you wait for thumbs/files to download
• a new include_blurhash parameter will include the blurhash when only_return_basic_information is true
• file_metadata also shows the file's pixel_hash now. the algorithm here is proprietary to hydrus, but you can throw it into 'system:similar files' to find pixel dupes. I expect to add perceptual hashes too
• the help is updated to talk about this
• I updated the unit tests to deal with this
• the error when the api fails to parse the client api header is now a properly handled 400 (previously it was falling to the 500 backstop)
• the client api version is now 53

misc

• I'm sorry to say I'm removing the Deviant Art artist search and login script for all new users, since they are both broken. DA have been killing their nice old API in pieces, and they finally took down the old artist gallery fetch. :(. there may be a way to finagle-parse their new phone-friendly, live-loading, cloud-deployed engine, but when I look at it, it seems like a much bigger mess than hydrus's parsing system can happily handle atm. the 'correct' way to programatically parse DA is through their new OAuth API, which we simply do not support. individual page URLs seem to still work, but I expect them to go soon too. Sorry folks, try gallery-dl for now--they have a robust OAuth solution
• thanks to a user, we now have 'epub' ebook support! no 'num_words' support yet, but it looks like epubs are really just zips with some weird metadata files and a bunch of html inside, so I think this'll be doable with a future hacky parser. all your existing zip files wil be scheduled for a metadata rescan to see if they are actually epubs (this'll capture any secret kritas and procreates, too, I think)
• the main UI-level media object is now aware of a file's pixel hash. this is now used in the duplicate filter's 'these are pixel duplicates' statements to save CPU. the jank old on-the-fly calculation code is all removed now, and if these values are missing from the media object, a message will now be shown saying the pixel dupe status could not be determined. we have had multiple rounds of regen over the past year and thus almost all clients have full database data here, so fingers crossed we won't see this error state much if at all, but let me know if you do and I'll figure out a button to accelerate the fix
• the thumbnail right-click->open->similar files menu now has an entry for 'open the selection in a new duplicate filter page', letting you quickly resolve the duplicates that involve the selected files
• pixel hash and blurhash are now listed, with the actual hash value, in the share->copy->hash thumbnail right-click menu
• thanks to a user, 'MPO' jpegs (some weird multi-picture jpeg that we can't page through yet) now parse their EXIF correctly and should rotate on a metadata-reparse. since these are rare, I'm not going to schedule a rescan over everyone's jpegs, but if you see a jpeg that is rotated wrong, try hitting manage->regenerate->file metadata on its thumbnail menu
• I may have fixed a rare hang when highlighting a downloader/watcher during very busy network time that involves that includes that importer
• added a warning to the 'getting started with installing' and 'database migration' help about running the SQLite database off a compressed filesystem--don't do it!
• fixed thumbnail generation for greyspace PSDs (and perhaps some others)

boring cleanup

• I cleaned some code and added some tests around the new blurhash tech and thumbs in general
• a variety of metadata changes such as 'has exif', 'has icc profile' now trigger a live update on thumbnails currently loaded into the UI
• cleaned up some old file metadata loading code
• re-sorted the job list dropdown in the file maintenance dialog
• some file maintenance database work should be a bit faster
• fixed some behind the scenes stuff when the file history chart has no file info to show