kobotoolbox/kpi 2.026.37c

on GitHub

attachments: [breaking] recount drifted storage counters (#7696)

Storage usage that drifted away from the files actually stored,
sometimes below zero, gets repaired.

#7680 stopped storage usage from drifting, but did not repair the
accounts where it had already drifted. Their projects are now recounted
from the files they actually hold, in the background, so their storage
usage matches what they really store again. Some accounts may then show
more storage than before, and others less.

👷 Description for instance maintainers

A new partial index on attachments lets the recount find the projects
that may have drifted quickly. With SKIP_HEAVY_MIGRATIONS, migration
logger/0056 only prints the SQL to run by hand:

CREATE INDEX CONCURRENTLY "attach_inactive_xform_id_idx"
 ON "logger_attachment" ("xform_id")
 WHERE "delete_status" IS NOT NULL;

A new periodic task, recount-drifted-storage-counters, recounts these
projects in turns and starts over once done.

Storage recounts no longer suspend submissions. The
fix-stale-submissions-suspended-flag task, which unlocks accounts left
suspended (e.g. by a worker of the previous version stopped during the
deployment), now runs every 15 minutes on the low priority queue instead
of on Sunday nights only.

⚠️ Known issues: Submission suspensions can be lifted while
populate_submission_counters recalculates user's counters. This will
be fixed in another PR.

⚠️ Breaking change: in addition to what #7680 says about it,
update_attachment_storage_bytes is rewritten. It only recounts project
counters, from the attachments they actually hold, without suspending
submissions. --sync, --no-lock and --chunks are gone, and
--force now means "start over". Usage:

• ./manage.py update_attachment_storage_bytes: recount the projects
  that may have drifted (same selection as the task)
• --username <username>: recount every project of one user
• --all: recount every project, trashed ones included
• --dry-run: only report the drifts
• --force: start over instead of resuming an interrupted run with the
  same options
• -v 2: show progress

openrosa: remove misleading v1 deprecation warning from HTML responses (#7701)

Remove the misleading "v1 is deprecated" warning shown when opening
OpenRosa endpoints in a browser.

Opening an OpenRosa endpoint such as /submission in a browser now
shows the same simple page as the API v2, with a link to the OpenRosa
API documentation, instead of a warning about the removed v1 API.