kobotoolbox/kpi 2.026.33

on GitHub

KMLExport: cleanup legacy KoboCAT KML logic (#7299)

Removed legacy KML export generation from KoBoCAT in favor of routing
through our KPI Exports.

assetVersion: add version_number to /api/v2/assets/<uid>/versions/ (#7300)

This PR adds a version_number field to the asset versions API,
returning "12" for deployed versions and "11.4" for undeployed versions
(numbered relative to the deployment that precedes them), computed
server-side so the numbering stays correct regardless of pagination.

Why this needed to be done in the backend

The Form History UI (Project → Form tab) needs to label each version
like v12, v11.4, etc. - deployed versions get a whole "major" number,
and undeployed (draft) versions get a "minor" number counting up since
the last deployment.

The frontend currently derives this from array index, which only works
because the full list is loaded at once. We're moving this list to
paginated infinite scroll, and a version's correct number depends on the
entire history of the asset before it - not just the versions on the
current page. The frontend has no way to know, from a single page of
results, how many deployments happened earlier in history, so it can't
reliably compute this itself once pagination is introduced.

What changed

Added a version_number field to the versions serializer. For each
request, we fetch the asset's full version history as a lightweight (id,
deployed) list in a single ordered query, then walk it once in Python to
build a map of version id → label: deployed versions increment the major
number and reset the minor, undeployed versions increment the minor. The
serializer looks up each row's label from that map, so both the
paginated list endpoint and retrieve return identical,
pagination-independent numbers with no per-row queries.

Why computed on the fly instead of stored on the model

I considered storing these numbers as columns on AssetVersion, populated
at save time like the existing _content_hash field, but ruled it out
because the table has millions of rows and would need a long, risky
backfill migration for a display-only field. Version counts per asset
are in the low hundreds in practice, so fetching two integer columns for
one asset's history and looping over them is negligible work - it
requires no migration and is automatically correct for all existing
data. If per-asset version counts ever grow into the tens of thousands,
the stored-column approach becomes worth revisiting; at current scale it
isn't.

bulkProcessing: disable bulk option if no data (#7325)

The Transcribe selected audio files and Translate selected transcriptions options in the data table column menu are now greyed out
when none of the selected submissions has anything to work with.

bulkProcessing: consistent audio duration in UI (#7320)

Audio durations in Project → Data → Table now match the durations used
by bulk transcription and translation.

bulkProcessing: disable bulk approve option if nothing to approve (#7322)

Disable "Approve all selected" option if none of selected submissions
has things to approve.

bulkProcessing: improve UI blocking code based on bulk job status(es) (#7346)

Fixed transcripts and translations being locked for review/save while
other rows from the same bulk job were still being processed.

bulkProcessing: improve count for approving (#7351)

The Approve all selected dialog now counts only the transcripts or
translations that still need approval, and tells you how many of your
selected submissions were already approved.

dataTable: deduplicate non-audio attachment columns (#7268)

Data Table now safely de-duplicates stale attachment columns not only
for audio, but also for image, video, and file questions, so users no
longer see duplicate columns after group/path changes.

dataTable: freeze and hide for all (#7343)

Users without editing permissions can now hide, show, and freeze columns
in Project → Data → Table for their own session, without changing what
anyone else sees.

designSystem: use mantine Tabs component and deprecate our old component (#7293)

Replace old tabs component with updated one

designSystem: update project top tabs component (#7335)

Update project top tabs to new component.

designSystem: adjust Switch colors (#7398)

frontend: improve Select and MultiSelect UX (#7353)

Make all Selects and MultiSelects searchable by default. Also add
PageUp, PageDown, Home, and End keyboard navigation to them.

frontend: Make MultiSelect searchable by default (#7397)

Every place that uses MultiSelect in UI will now allow searching for
options.

massEmails: add llm requests to mass email queries (#7303)

Add a Mass email config for sending emails to users who are above
various thresholds in LLM usage.

massEmails: get rid of add to send option (#7342)

Remove the 'Add to daily send' option from the Django admin list page
for mass email configs.

openRosa: block superuser form submissions by default (#7371)

Superuser accounts can no longer submit data to forms, preventing
accidental damage to projects from test submissions.

Submitting data while signed in as a superuser can corrupt a project's
data. Superuser accounts are now blocked from submitting to any form —
whether through KoboCollect, Enketo web forms, or the API. Regular
accounts are unaffected; use one to submit or test data.

👷 Description for instance maintainers

This restriction is on by default. Self-hosters who deliberately
rely on superuser submissions can re-enable them by setting the
environment variable ALLOW_SUPERUSER_SUBMISSIONS=True. Doing so
re-arms a known footgun — superuser submissions can break projects.

organizations: org members api needs to expose sso status (#7328)

Add a new user__has_sso_enabled field to organizations member serializer
and updated the open api docs

processing: auto translation polling (#7244)

Automatic translations now use per-language polling in the Translations
tab. If a translation is still processing, the tab shows an in-progress
screen and keeps polling until the result is available. This also works
when users navigate away and come back.

projectDownloads: migrate KML to non-legacy (#7256)

KML exports now use the modern downloads flow (same as CSV/XLS/GeoJSON)
instead of the legacy iframe path.

projectDownloads: show media checkbox for GeoJSON (#7296)

GeoJSON exports now include media URLs by default and have a matching
"Include media URLs" checkbox under Advanced options, just like XLS and
CSV exports.

projectHistoryLogs: log when users view data (#7339)

Log when users view data for a project.

Create new project history logs whenever a user hits the
/api/v2/assets/{uid}/data/ endpoint.

👷 Description for instance maintainers

This is likely to add a ton more audit logs. It shouldn't be enough to
affect performance but it's worth noting.

stripe: add webhook handling to update status of unpaid subscriptions (#7254)

Update our Stripe billing integration so that certain subscription plans
(like Teams) can remain in an "unpaid" status indefinitely instead of
automatically canceling after payment retries fail.

stripe: only run cleanup tasks on stripe-enabled instances (#7395)

userReports: add LLM usage to user reports (#7386)

Add usage summaries for LLM requests to the user reports endpoint.

admin: correct CORS configuration guidance (#7284)

admin: allow creating ExtraUserDetail without a manual uid (#7317)

Superusers can now add a user's extra details from the Django admin
without having to invent an ID by hand.

Adding a new "extra user detail" record from the Django admin was
effectively impossible. The form demanded a uid that the admin was
supposed to generate automatically, and it rejected the empty {} value
for the data field even though that is the field's own default. Both
fields now behave as expected: the uid is generated on save and is no
longer editable, and {} is accepted for data.

attachments: use root_uuid for media folder naming (#7394)

Media files for edited submissions could end up in an export folder that
no longer matches the submission, making them hard to find.

bulkProcessing: hide transcript language from bulk translation modal (#7403)

The language a transcript is already in is no longer offered when
creating a translation, so you can't accidentally translate English into
English and end up with a blank column you can't get rid of.

bulkProcessing: translations tab alert (#7402)

The "this submission is already being processed by another job" warning
now also appears on the Translations tab, not just on the Transcript
tab.

darker: fail the linter job when darker crashes (#7316)

No user-facing change — this fixes an internal code-quality check that
was reporting success even when it had not actually run.

The automated Python style check that runs on every code change could
fail to start and still report a green result, so problems it was meant
to catch could slip through unnoticed. It now reports a clear failure
whenever it cannot complete.

dataTable: improve select_x data displaying (#7362)

Data Table now shows all selected options of a select_many question,
falling back to the raw value for options that were renamed or removed
in a later version of the form.

datatable: fallback values for missing choice names (#7381)

Fixes the select one/many column filter dropdown on the data table to
handle choices with a missing name by falling back to $autoname then
$autovalue before dropping value from list.

digestAuth: use a fixed content type on auth failure (#7352)

Fixed a server error (HTTP 500) that could occur when authentication
failed while downloading a submission attachment.

When a client tried to download a submission attachment (for example
over ODK Briefcase) with invalid credentials, the server could respond
with an unexpected error instead of a proper "authentication required"
response. It now consistently returns a 401.

formbuilder: preserve manual skip logic and validation inputs (#7347)

Fixed an issue where skip logic and validation criteria entered manually
could be lost after closing and reopening the question settings panel.

formbuilder: crash with unsupported type (#7392)

Fixed Form Builder failing to open forms that contain question types it
can't edit, and no longer dropping those questions when the form is
saved.

Some valid XLSForm question types have no editor in Form Builder.
Opening such a form used to break the editor, and saving it - silently
deleted the question.

Now those questions show up as a card explaining that they can't be
edited here and will be left alone when you save. The rest of the form
works normally. Affected types include select_one_external, email,
osm, percentage, phone number, number of days in last month/six months/year, q select, q select1, and the uri:* metadata types.

A card:

frontend: notification with multiple error lines layout (#7297)

Improve layout of import error notification when multiple lines of text
needs to be displayed.

imports: validate URLs against SSRF before fetching them (#7520)

Importing a project from a URL now refuses addresses that point back
inside the server's own network.

When you import a project by pasting a URL, KoboToolbox fetches that
address for you. Until now it would fetch any address, including ones
that only exist inside the server's private network. Those are now
refused and the import stops with a clear error message. The same check
applies to form media added by URL. Imports and media from ordinary
public web addresses work exactly as before.

massEmails: migration conflicts (9fb6a98)

massEmails: add missing autoqa eligibility checks after .30→.33 merge (#7457)

Fixes a gap left over from the release/2.026.30 → release/2.026.33
merge: three mass-email queries had no matching eligibility check.

release/2.026.33 already had AutoQA usage mass-email queries
(users_above_{80,90,100}_percent_autoqa_usage) that predate
release/2.026.30's stale-record eligibility recheck (), so
those three queries had no matching single-user eligibility check once
the branches merged.

openapi: type validation error responses as ErrorValidation (#7275)

The API reference now documents validation errors with their real shape
— a map of field name to error messages — instead of an untyped
placeholder.

When a request fails validation, the KoboToolbox API returns each
field's errors as a list of messages. Until now the API reference
described this 400 response as an untyped detail object, which
didn't match what the API actually sends. It is now documented
accurately, so anyone building against the API — or relying on the types
generated from its schema — gets the correct shape for validation
errors.

organizations: return displayable error details for invites and members (#7332)

Error messages shown when managing team or organization members now
explain what actually went wrong, instead of a generic "there was an
error" message.

Some screens in the Members and Usage sections replaced the
server's explanation with their own wording — changing the role on an
invitation that had already been accepted just said "There was an error
updating this invitation." They now show the specific reason. When the
server can't be reached at all, a single plain message is shown instead
of a technical status code.

profile: dropdown z-index (#7304)

Fixes a bug that resulted in the profile dropdown from the nav header
being hidden by other components.

projectDownloads: kml exporting bug (#7442)

Fixed an error that blocked KML exports in projects where a GeoJSON
export was created earlier.

Options that apply to a single export format no longer follow you when
you switch format. Before this fix, exporting GeoJSON and then KML sent
the GeoJSON-only "Flatten GeoJSON" option along with the KML request,
and the server refused it. Projects already in that state recover on
their next export, nothing to clean up.

projectOwnership: stop false transfer failures and improve admin transfer details (#7294)

Project ownership transfers are no longer reported as failed when the
transfer actually completed successfully.

Transferring projects could end as "failed" even though everything had
moved correctly — a single already-deleted file among thousands was
enough. A file that no longer exists cannot be moved, so it no longer
counts as a failure, and the same now applies when an internal retry
re-runs a step that had already finished. Genuine problems are still
reported, and a new log page shows what happened for a given project.

projectViews: allow filtering by owner organization (#7283)

qualitativeAnalysis: allow duplicate casing in QA tags questions (#7298)

Updated tag duplication logic to allow different casings of the same tag
to be added (and suggested) as separate tags.

1. ℹ️ have an account and a project with multiple audio submissions
2. add a tags QA question
3. add tags "Poverty" and "poverty"
4. 🔴 [on main] notice that "poverty" doesn't get added
5. go to a new submission
6. add tag "poverty" here
7. 🟢 [on main] notice that "poverty" gets added
8. go to a new submissions
9. try adding both "Poverty" and "poverty" from the tags suggestions
10. 🔴 [on main] notice that both tags don't get added
11. 🟢 [on PR] notice that both tags can be added

submissions: reject trailing slash on OpenRosa endpoints with 404 (#7361)

Calling an OpenRosa endpoint (form list or submission) with an
accidental trailing slash now returns a clear "not found" message
instead of a confusing security error.

The OpenRosa endpoints are meant to be called without a trailing slash.
Adding one (e.g. /submission/) used to surface a misleading CSRF
error. They now respond with an explicit 404 that names the correct URL
to retry. The official clients (Collect, Enketo) are unaffected — they
already build the correct slash-less URLs. This only helps people
writing their own integrations or curl commands.

👷 Description for instance maintainers

New OpenRosaTrailingSlashMiddleware, ordered ahead of
CsrfViewMiddleware, intercepts any request whose path ends in
/submission/ or /formList/ — covering the authenticated, per-user,
data-collector and asset-snapshot variants — and returns a 404 pointing
at the slash-less URL. Honoring the slash was deliberately rejected:
OpenRosa clients build slash-less URLs, and Django's APPEND_SLASH
redirect would drop the POST body on submissions.

tags: return each tag once from /api/v2/tags/ (#7315)

TagViewSet.get_queryset() filters across the taggit_taggeditem join
table, so a tag attached to N accessible assets was emitted N times,
.distinct() collapses it back to one row per tag.

TagViewSet.get_queryset() filters Tag on
taggit_taggeditem_items__* to restrict tags to assets the user may
view. Because that filter spans a multi-valued relationship, the
underlying SQL joins taggit_tag onto taggit_taggeditem and returns
one row per tag/asset pairing so a tag applied to two accessible assets
came back twice, with an identica name and url.

trashBin: restart trash bin tasks which failed on transient errors (#7390)

This PR makes trash bin deletion jobs that fail on a transient
infrastructure error retry themselves automatically, by leaving them
in-progress so the existing task restarter picks them up, and fixes
three pre-existing bugs in that restarter including one that deleted
accounts a superuser was supposed to approve first.

Problem

Trash bin deletion jobs (accounts, projects, attachments) that died on
an infrastructure error (MongoDB unreachable, a PostgreSQL deadlock, a
Celery time limit, an OOM kill) were marked FAILED and left there.
Nothing retried them, so half-deleted objects lingered until someone
noticed and restarted them by hand.

What this changes

When a job fails, its error is matched against a list of known-transient
patterns. If it matches, the object is left IN_PROGRESS instead of
FAILED which is exactly what the existing task_restarter already
looks for, so it gets restarted with no new machinery. Any other failure
still goes to FAILED and waits for a human.

Attempts are counted and capped: once an object exceeds
TRASH_BIN_MAX_AUTO_RESTARTS it is flagged FAILED and stops retrying, so
it can't loop forever and becomes visible to a human.

Three pre-existing bugs on main

1. The stuck check was effectively dead for started deletions. It
   required an object to be roughly twice its retention period old before
   being eligible. (e.g., 7-day-retention project failing on day 7 was only
   restarted on day 14; 360 days for accounts)
2. Accounts held for manual deletion were deleted automatically. Their
   scheduled time is computed into the past, and the restarter reads that
   timestamp rather than the disabled Celery schedule so it force-started
   the deletion ~75–105 minutes after trashing, voiding the "a superuser
   must approve this" guarantee.
3. A manual deletion that genuinely got stuck was never restarted. The
   mirror image: once retention is changed to a normal value, that same
   past timestamp fails the check, so a superuser-started deletion whose
   worker died sat half-complete in IN_PROGRESS for months.

The fix for all three: a deletion that is already in progress has by
definition passed its scheduled time, so that time is no longer
consulted for it. Only objects that never started still get the "is it
due yet?" check, and those are additionally never auto-started when
they're waiting for a superuser.

Made the AssetTagsModal and FormLanguagesManager Storybook
interaction tests more reliable so they stop failing at random in CI.

RESTServices: mantineify code (#7302)

Migrated REST Services feature to use Mantine components.

dataTable: mantineify table settings modal (#7288)

Migrated Table Settings modal to Mantine Modal, migrated the in-modal
component to TypeScript and made it a functional component that uses
Mantine components. Also fixed "Reset" button to use proper defaults.

dataTable: mantineify table media preview (#7289)

Migrates Table Media Preview modal and Text Modal to the Mantine modal
flow, removes the legacy BigModal path for table media preview, and adds
stable offline-safe DataTableCell stories for media behaviors.

frontend: replace ToggleSwitch with Mantine Switch (#7387)

Using Mantine Switch throughout the app, which has a better colors than
our old component.

frontend: switch KoboDropdown instances to Mantine Menu (#7380)

The sorting and options menus in the project list, per-project usage
table, data table and qualitative analysis now use Mantine Menu.

map: remove leaflet-omnivore (#7301)

Replaces defunct leaflet-omnivore package with more targeted
dependencies for parsing map overlay data, while also removing support
for .wkt files in this context.

myProjects: replace react-infinite-scroller with in-house solution (#7391)

Scrolling through a long list of projects now loads the next batch more
reliably, and a failed load offers a Retry button instead of just an
error message.

permissions: mantineify sharing modal (#7295)

Sharing settings were modernized to a Mantine modal flow, with clearer
public/anonymous sharing controls, better form validation feedback, and
accessibility fixes.

projectDownloads: migrate to orval (#7250)

dependencies: upgrade fantasticon to 4.1 (#7321)

dependencies: upgrade immutable package (#7323)

deps: bump formpack pin for missing-value NLP export fix (#7345)

frontend: rename route file (#7382)

frontend: fix button text casing around Data Table (#7409)

Fix few buttons to use Sentence case instead of lower case or Title Case. Also changed bulk related text to more sensible one.

node: drop support for node 20 (#7308)

orval: upgrade Orval to 7.21 (#7305)

upgrade drf-spectacular (#7324)

pull transifex translations for 2.026.33 (17d6049)