hydrusnetwork/hydrus v574

Version 574

on GitHub

local hashes cache

• we finally figured out the 'update 404' issue that some PTR-syncing users were getting, where PTR processing would halt with an error about an update file not being available on the server. long story short, SQLite was sometimes crossing a wire in the database on a crash, and this week I add some new maintenance code to fix this and catch it in future
• the local hash cache has a bunch of new resync/recovery code. it can now efficiently recover from missing hash_ids, excess hash_ids, desynced hash_ids, and even repopulate the master hash table if that guy has missing hash_ids (which can happen after severe db damage due to hard drive failure). it records all recovery info to the log
• the normal database->regenerate->local hashes cache function now works entirely in this new resync code, making it significantly faster (previously it just deleted and re-added everything). this job also gets a nicer popup with a summary of any problems found
• when the client recovers from a bad shutdown, it now runs a quick sync on the latest hash_ids added to the local hashes cache to ensure that desync did not occur. fingers crossed, this will work super fast and ensure that we don't get the 404 problem (or related hash_id cross-wire problems) again
• on repository processing failure and a scheduling of update file maintenance, we now resync the update files in the local hash cache, meaning the 404 problem, if it does happen again, will now fix itself in the normal recovery code
• on update, everyone is going to get a full local hash cache resync, just to catch any lingering issues here. it should now work super fast!
• fixed an issue where the local hash and tags caches would not fully reset desynced results on a 'regenerate' call until a client restart

misc

• thanks to a user, the default twitter downloader I added last week now gets full-size images. if you spammed a bunch of URLs last week, I apologise: please do a search for 'imported within the last 7 days/has a twitter url/height=1200px' and then copy/paste the results' tweet URLs into a new urls downloader. because of some special twitter settings, you shouldn't have to set 'download the file even if known url match' in the file import options; the downloader will discover the larger versions and download the full size files with no special settings needed. once done, assuming the file count is the same on both pages, go back to your first page and delete the 1200px tall files. then repeat for width=1200px!
• the filetype selector in system:filetype now expands to eat extra vertical space if the dialog is resized
• the filetype selector in file import options is moved a bit and also now expands to eat extra vertical space
• thanks to a user, the Microsoft document recognition now has fewer false negatives (it was detecting some docs as zips)
• when setting up an import folder, the dialog will now refuse to OK if you set a path that is 1) above the install dir or db dir or 2) above or below any of your file storage locations. shouldn't be possible to set up an import from your own file storage folder by accident any more
• added a new 'apply image ICC Profile colour adjustments' checkbox to options->media. this simply turns off ICC profile loading and application, for debug purposes

boring cleanup

• the default SQLite page size is now 4096 bytes on Linux, the SQLite default. it was 1024 previously, but SQLite now recommend 4096 for all platforms. the next time Linux users vacuum any of their databases, they will get fixed. I do not think this is a big deal, so don't rush to force this
• fixed the last couple dozen missing layout flags across the program, which were ancient artifacts from the wx->Qt conversion
• fixed the WTFPL licence to be my copyright, lol
• deleted the local booru service management/UI code
• deleted the local booru service db/init code
• deleted the local booru service network code
• on update, the local booru service will be deleted from the database