Hey, we had trouble with the macOS build this week, so we had to try a different method. If you are a macOS user, please grab it here
quality of life
- when subscriptions hit their 'periodic file limit', which has always been an overly technical term, the popup message now explains the situation in better language. it also now provides a button to automatically fill in the gap via a new gallery downloader page called 'subscription gap downloaders' that gets the query with a file limit five times the size of the sub's periodic download limit
- I rewrote the logic behind the 'small initial sync, larger periodic sync' detection in subscription sync, improving url counting and reliability through the third, fourth, fifth etc... sync, and then generalised the test to also work without fixed file limits and for large-gallery sites like pixiv, and any site that has URLs that often produce multiple files per URL. essentially, subs now have a nice test for appropriate times to stop url-adding part way through a page (typically, a sub will otherwise always add everything up to the end of a page, in order to catch late-tagged files that have appeared out of order, but if this is done too eagerly, some types of subs perform inefficiently)
- this matters for PTR accounts: if your repository account does not have permissions to upload something you have pending, the popup message talking about this now hangs around for longer (120 seconds), explains the issue better, and has a button that will take you directly to the manage services panel for the service and will hit up 'check for auto-account creation'
- in manage services, whenever you change the credentials (host, port, or access key) on a restricted service, that service now resets its account to unknown and flags for a swift account re-fetch. this should solve some annoying 'sorry, please hit refresh account in review services to fix that manually' problems
- a new option in maintenance and processing allows you to disable idle mode if the client api has had a request in the past x minutes. it defaults disabled
- an important improvement to the main JobScheduler object, which farms out a variety of small fast jobs, now massively reduces Add-Job latency when the queue is very busy. when you have a bunch of downloaders working in the background, the UI should have much less lag now
- the options->speed and memory page has a full pass. the thumbnail, image, and image tile caches now have their own sections, there is some more help text, and the new but previously hardcoded 10%/25% cache and prefetch limits are now settable and have dynamic guidance text that says 'about a 7,245x4,075 image' as image cache options change
- all the cache options on this page now apply instantly on dialog ok. no more client restart required!
other stuff, mostly specific niche work
- last week's v441->442 update now has a pre-run check for free disk space. users with large sessions may need 10GB or more of free space to do the conversion, and this was not being checked. I will now try to integrate similar checks into all future large updates
- fixed last week's yandere post parser link update--the post url class should move from legacy moebooru to the new yandere parser correctly
- the big maintenance tasks of duplicate file potentials search and repository processing will now take longer breaks if the database is busy or their work is otherwise taking a long time. if the client is cluttered with work, they shouldn't accidentally lag out other areas of the program so much
- label update on ipfs service management panel: the server now reports 'nocopy is available' rather than 'nocopy is enabled'
- label update on shortcut: 'open a new page: search page' is now '...: choose a page'
- fixed the little info message dialog when clicking on the page weight label menu item on the 'pages' menu
- 'database is complicated' menu label is updated to 'database is stored in multiple locations'
- options->gui pages->controls now has a little explanatory text about autocomplete dropdowns and some tooltips
- migrate database dialog has some red warning text up top and a small layout and label text pass. the 'portable?' is now 'beneath db?'
- the repositery hash_id and tag_id normalisation routines have two improvements: the error now shows specific service_ids that failed to lookup, and the mass-service_hash_id lookup now handles the situation where a hash_id is mapped by more than one service_id
- repository definition reprocessing now corrects bad service_id rows, which will better heal clients that previously processed bad data
- the client api and server in general should be better about giving 404s on certain sorts of missing files (it could dump out with 500 in some cases before)
- it isn't perfect by any means, but the autocomplete dropdown should be a little better about hiding itself in float mode if the parent text input box is scrolled off screen
- reduced some lag in image neighbour precache when the client is very busy
boring code cleanup
- removed old job status 'begin' handling, as it was never really used. jobs now start at creation
- job titles, tracebacks, and network jobs are now get/set in a nicer way
- jobs can now store arbitrary labelled callable commands, which in a popup message becomes a labelled button
- added some user callable button tests to the 'make some popups' debug job
- file import queues now have the ability to discern 'master' Post URLs from those that were created in multi-file parsing
- wrote the behind the scenes guts to create a new downloader page programmatically and start a subscription 'gap' query download
- cleaned up how different timestamps are tracked in the main controller