endurain-project/endurain v0.20.0

on GitHub

v0.20.0 - Event-driven activities, background jobs and scale-out

⚠️ Warning

• The DB has schema changes (a large set of new migrations). Backup your DB and data folder before updating; restoring that backup is the only way back to v0.19.x.
• This release contains breaking API changes (see below). Third-party clients and integrations using the REST API will need updating.
• Review the environment variable changes below. Endurain refuses to start while RATE_LIMIT_STORAGE_URI or AUTH_SECURITY_STORAGE_URI is still set.

Highlights

The backend foundation for this release is tracked in #785; related PRs are linked below.

• Activity pipeline rebuilt on an event-driven core. Activities now publish events (activity.created, ...) and independent subscribers react to them: map thumbnails, reverse geocoding, notifications, source-file retention. Failures in one subscriber no longer affect the request or the others, and scheduled backfills repair anything missed.
• Durable background jobs (#796). Optional job queue (JOBS_ENABLED=true) with a transactional outbox, retries with backoff, leases and a dead-letter state. File uploads, bulk import, provider refresh (Strava/Garmin) and email delivery now run as jobs.
• Ready to scale out (#771). New deployment profiles (local / distributed), shared Redis state, S3-compatible storage, PostgreSQL advisory locks for scheduled jobs, multiple web workers and a dedicated worker container (APP_ROLE=worker). The default single-container setup needs no changes. See Scaling and background jobs in the docs.
• Platform package. Storage, events, jobs, geocoding, mail and migrations now come from the jasil package. Apprise is gone.
• Hardened security, see below.
• Timezone-correct stats (#910). Week/month boundaries, gear component dates and health data now use the user's configured timezone and week start; activities without a GPS track use the user timezone too.
• Time in zone on activities (#789). The heart rate zones card now shows the time spent in each zone alongside the percentage.

Import folders moved (action required if you import from disk)

Folder-based imports are now isolated per user, and the Strava export layout changed:

• Bulk import: data/activity_files/bulk_import/ becomes data/activity_files/bulk_import/<your user id>/.
• Strava export: data/activity_files/strava_import/ (with activities.csv, activities/, media/) becomes data/activity_files/strava_import/<your user id>/, or just upload the export ZIP from the Import page.
• Files that fail to import are no longer left in bulk_import/import_errors; they are kept in the bulk_import_errors storage area under <user id>/<import id>/<file name>.

Files already sitting in the old locations are not picked up; move them before starting an import. Details in the Importing data docs page.

Email

Apprise was removed; email now uses the jasil mail provider (#825).

• Event-driven delivery. Password reset, sign-up verification, admin approval request (one per admin) and sign-up approved emails are published as events (auth.password_reset_requested, auth.sign_up_verification_requested, auth.sign_up_approval_requested, auth.sign_up_approved) and handled by subscribers. With JOBS_ENABLED=true they run as durable jobs with retries and backoff; otherwise delivery is best-effort on the in-process event bus (no retries, lost on restart).
• Asynchronous sending. The API no longer reports delivery failure: the 500 Unable to send password reset email response and the "Failed to send verification email" sign-up message are gone. Failures show up in the logs and, with jobs enabled, in the Jobs dashboard. Permanent failures (SMTP 5xx, email not configured, invalid message) are logged and not retried; transient ones (connection, TLS, 4xx) are retried when jobs are enabled.
• Only the newest reset or verification link works. Requesting a new password reset or sign-up verification email invalidates the previous unused link. Before, every unexpired link stayed valid.
• No secrets in events. Event payloads carry user ids only; the reset/verification token is created by the handler and committed before the email is sent, so no token or address is stored in events or jobs.
• Test email from the admin settings. New Email card on Settings > Server shows whether the SMTP settings are valid (and, when they are not, which settings are at fault, by name only) and a Send test email button. The test is sent synchronously to the admin's own address, never to an arbitrary recipient, and a failure shows the mail server's failure category and SMTP status code (for example "authentication failed (status 535)"). API: GET /server_settings/email (server_settings:read) and POST /server_settings/email/test (server_settings:write, rate limited; 503 when email is not configured, 502 when the mail server refuses the message). The email text is English only for now; other languages fall back to English.
• Clear startup log. One info line when SMTP_HOST is unset, one warning per problem when the SMTP settings are invalid (email disabled), one info line when enabled.

Environment Variables & Configuration

New Variables

• DEPLOYMENT_PROFILE (default: local) - local, distributed or custom. Endurain refuses to start on an inconsistent topology and names the variable at fault.
• WEB_WORKERS (default: 1) - number of web-server processes. Above 1 requires REDIS_URL.
• REDIS_URL (default: empty) - default DSN for every Redis-backed capability (rate limits, login lockout, pending MFA, websocket tickets, event bus).
• STATE_URI / EVENTS_URI / LOCK_URI / STORAGE_URI - fine-grained overrides. STORAGE_URI accepts local:// (default) or s3://bucket/prefix.
• JOBS_ENABLED (default: false) plus JOBS_RUN_IN_PROCESS_WORKER, JOBS_MAX_ATTEMPTS, JOBS_BACKOFF_BASE_SECONDS, JOBS_BACKOFF_MAX_SECONDS, JOBS_LEASE_SECONDS, JOBS_BATCH_SIZE, JOBS_POLL_INTERVAL_SECONDS, JOBS_RETENTION_DAYS - durable job queue tuning.
• APP_ROLE (default: api) - worker runs a dedicated job-worker container from the same image.
• EVENT_LOG_ENABLED / EVENT_LOG_RETENTION_DAYS (defaults: true / 90).
• Docker build arg PYTHON_EXTRAS=distributed - installs the Redis/S3 client libraries. The default image does not include them; if you use REDIS_URL or s3:// storage, build your own image with this extra.

Changed

• SMTP_FROM - Now falls back to SMTP_USERNAME when unset (it was auto-detected by Apprise before). If neither is set, email is disabled with a startup warning. A bare address is sent as Endurain <address>; Name <address> is kept as written.
• SMTP_SECURE - false (no TLS) is now only accepted for a private, loopback or link-local SMTP_HOST, and not together with SMTP_USERNAME/SMTP_PASSWORD. Otherwise email is disabled with a warning. If you relied on unencrypted authenticated SMTP, switch to starttls or ssl.
• SMTP_HOST / SMTP_PORT / SMTP_USERNAME / SMTP_PASSWORD - validated at startup: SMTP_HOST must be a hostname or IP address (no scheme, port or path), SMTP_PORT 1-65535, and username and password must be set together and contain only ASCII characters. An invalid combination disables email with one warning per problem naming the setting (never its value); Endurain still starts.
• TRUSTED_PROXIES - when BEHIND_PROXY=true, make sure it lists your reverse proxy(ies): the client IP is now the first X-Forwarded-For hop, read right to left, that is not a trusted proxy.

Removed

• RATE_LIMIT_STORAGE_URI and AUTH_SECURITY_STORAGE_URI - replaced by REDIS_URL / STATE_URI. Endurain refuses to start while either is still set, so replace them with REDIS_URL in your compose file before updating.
• JAEGER_ENABLED, JAEGER_PROTOCOL, JAEGER_HOST, JAEGER_PORT - OpenTelemetry/Jaeger tracing has been removed, along with the Jaeger service in the example compose files.

Other

• New data folders are created at start: activity_files/upload_incoming and activity_files/upload_staging.
• The Docker compose examples are simplified: the multiple-backends example is removed and Redis is no longer part of the default examples (REDIS_URL is commented out in .env.example and DEPLOYMENT_PROFILE is not set there). Add Redis yourself only for a distributed deployment.

Breaking API changes

If you only use the Endurain web UI you do not need to do anything. API consumers, note the main route changes (full list in the OpenAPI reference):

Authentication routes (login, refresh, MFA) are unchanged in this release.

General changes:

• Paginated list envelopes. Activity, follower, lap, set, workout-step and stream list endpoints return a paginated envelope with totals instead of bare arrays. The activity feed uses keyset pagination. Empty results are [], never null.
• Nested child resources. Activity child resources (laps, sets, streams, workout steps, media, exercise titles) live under /activities/{id}/....
• 202 Accepted for uploads and provider refresh. File upload and Strava/Garmin refresh now return 202 and run as durable jobs. Uploads support an Idempotency-Key header.
• PATCH for bulk activity edit and activity ETags.
• RFC 9457 problem documents for all error responses (replaces {"detail": ...}), documented in the OpenAPI schema.
• Signed, token-gated URLs for activity media, thumbnails and user profile photos. Tokens expire; fetch fresh URLs from the API rather than caching them.
• Followers API reworked (new model, request/accept notifications, privacy-aware lists; unknown fields in follow decisions are rejected).
• Activity responses use Read schemas; timezone formatting fields were removed from activity and lap schemas. Datetimes are normalised to UTC.
• Public server settings now expose the configured ALLOWED_REDIRECT_SCHEMES.
• Thumbnails are now WebP, generated through the event bus and storage provider (#774). Existing PNGs are migrated.

Security

This release fixes the following advisories:

Thank you to everyone who reported these responsibly.

Additional hardening:

• Visibility masking for activity metrics and private notes based on viewer ownership; visibility policy for live Strava API activities; private-profile exports enforced; owner-scoped bulk reads.
• Rate limits are now keyed per caller (JWT signature verified), with new limits on file uploads and follower actions.
• Spoofed X-Forwarded-For hops are ignored (#829). A client could prepend a forged leftmost entry to bypass rate limiting and poison session IPs. The header is now read right to left and the first hop that is not a trusted proxy is the client.
• SSRF protection with async DNS resolution for map-tile rendering and other outbound requests; Stadia URL validation; HTTPS enforced for tile API keys. The hostname lookup pool is bounded (at most 32 running or queued lookups), so a stalled resolver can no longer pile up requests: further callers get an immediate 503 with Retry-After, and the pool is shut down cleanly on exit (#833).
• Safer uploads: all UploadFile handling goes through a single module, user filenames are never used for paths, bulk-import rejects symlinks and is isolated per user.
• Control-character and path-encoding validation on redirects; log fingerprinting for sensitive values.
• Replaced bleach with nh3; migrated httpx to httpx2.
• Content-Security-Policy now includes object storage origins for blobs.

Backend

• Backend restructured into modules with enforced boundaries; services own transactions and publish events.
• Unified activity ingestion (ActivityCore) with content-hash and dedup_key deduplication, UTC normalisation and idempotent re-imports; activity media deduplicated by content_hash.
• Activity source-file storage, a pluggable file-parser registry and orphan sweeps for files, media and thumbnails.
• Reverse geocoding with location backfill and defer logic for unresolved activities.
• HR zone percentages are no longer stored; they are computed from your current max heart rate when streams are read.
• Thumbnail regeneration with retry scheduling.
• Distributed locking for scheduled jobs; user activity locking during deletion; bulk-deletion events.
• Follower invariants enforced with check constraints; activity exercise-title uniqueness.
• Legacy timezone aliases canonicalised.
• Notifications are created idempotently from source event IDs.
• Week start respected in stats (#910); the week feed, week/month stats, summaries and goals results now read the user once per request (#835).
• Pruning of event_log, processing_jobs and event_outbox in bounded batches.
• Email requests from async routes run in the thread pool so they no longer block the event loop.
• Garmin sleep sync no longer overwrites existing data (#830). For a date without sleep, Garmin returns a record with every field empty; the sync treated it as real data and overwrote any existing sleep row for that date, including rows written by another source. Such dates are now skipped, and the sync only updates rows that Garmin wrote.
• OpenStreetMap default tile URL (#812). The default and the OpenStreetMap preset now use https://tile.openstreetmap.org/{z}/{x}/{y}.png (no a/b/c subdomains), as OSM's tile usage policy asks. A migration rewrites a stored legacy OSM URL; custom tile servers are left unchanged. Fixes #811.
• Dependency updates: FastAPI, SQLAlchemy, Alembic, Pydantic, psycopg, uvicorn, timezonefinder, garminconnect and more. TypeScript stays on 6.0 because TypeScript 7 does not work with vue-tsc yet.
• Tooling and images: uv 0.12.x, astral-sh/setup-uv pinned by commit hash, refreshed digests for the node:24-slim and python:3.13-alpine base images.

Frontend

• Time spent in each heart rate zone shown in the heart rate zones card (#789).
• SSO redirect honors ALLOWED_REDIRECT_SCHEMES (#826). The frontend only allowed endurain:, so mobile SSO handoff to other apps (for example Gadgetbridge) never happened. It now validates redirects against the configured schemes; the allow-list is still deny-by-default and unsafe schemes are never accepted.
• Pending-thumbnail placeholders while map images are generated.
• Feed and search updated for paginated results.
• Gear component dates use calendar dates in the local timezone.
• Vite PWA precache excludes the large login background image.
• OpenStreetMap tile URL fallbacks updated to match the backend.

Docs

• New API reference (OpenAPI) page under Integrate: browse every REST endpoint, parameters and responses without installing anything. It is generated from the backend code at docs build time, so it always matches the release.
• New Nike Run Club export guide in the import docs (#819).
• New Map tile server section in Advanced setup explaining how the tile URL and the tile server API key are used by browser maps and by backend thumbnails, which providers receive the key in a header (Stadia Maps), and the workaround for providers that expect a different query parameter name (#831).
• New: scaling and background jobs, reverse proxy, operations, troubleshooting, export/import, error handling, API reference for Garmin/Strava, events and subscribers, activity ingestion, supported types.
• SMTP variable table updated (SMTP_FROM fallback, SMTP_SECURE=false restrictions, retries need JOBS_ENABLED); Apprise removed from the tech stack and feature lists.

Contributors

• @ycanerol in #812 and #819
• @DanyPM in #789