endurain-project/endurain v0.20.0-beta2

on GitHub

v0.20.0-beta2 - Event-driven activities, background jobs and scale-out

⚠️ Warning: beta release

• This is a pre-release. Do not run it against data you cannot restore.
• The DB has schema changes (a large set of new migrations). Backup your DB and data folder before updating; restoring that backup is the only way back to v0.19.x.
• This release contains breaking API changes (see below). Third-party clients and integrations using the REST API will need updating.
• Review the environment variable changes below.

Highlights

The backend foundation for this release is tracked in #785; related PRs are linked below.

• Activity pipeline rebuilt on an event-driven core. Activities now publish events (activity.created, ...) and independent subscribers react to them: map thumbnails, reverse geocoding, notifications, source-file retention. Failures in one subscriber no longer affect the request or the others, and scheduled backfills repair anything missed.
• Durable background jobs (#796). Optional job queue (JOBS_ENABLED=true) with a transactional outbox, retries with backoff, leases and a dead-letter state. File uploads, bulk import and provider refresh (Strava/Garmin) now run as jobs.
• Ready to scale out (#771). New deployment profiles (local / distributed), shared Redis state, S3-compatible storage, PostgreSQL advisory locks for scheduled jobs, multiple web workers and a dedicated worker container (APP_ROLE=worker). The default single-container setup needs no changes. See Scaling and background jobs in the docs.
• Platform package. Storage, events, jobs, geocoding and migrations now come from the jasil package.
• Hardened security, see below.
• Timezone-correct stats (#910). Week/month boundaries, gear component dates and health data now use the user's configured timezone and week start; activities without a GPS track use the user timezone too.

Import folders moved (action required if you import from disk)

Folder-based imports are now isolated per user, and the Strava export layout changed:

• Bulk import: data/activity_files/bulk_import/ becomes data/activity_files/bulk_import/<your user id>/.
• Strava export: data/activity_files/strava_import/ (with activities.csv, activities/, media/) becomes data/activity_files/strava_import/<your user id>/, or just upload the export ZIP from the Import page.
• Files that fail to import are no longer left in bulk_import/import_errors; they are kept in the bulk_import_errors storage area under <user id>/<import id>/<file name>.

Files already sitting in the old locations are not picked up; move them before starting an import. Details in the Importing data docs page.

Environment Variables & Configuration

New Variables

• DEPLOYMENT_PROFILE (default: local) - local, distributed or custom. Endurain refuses to start on an inconsistent topology and names the variable at fault.
• WEB_WORKERS (default: 1) - number of web-server processes. Above 1 requires REDIS_URL.
• REDIS_URL (default: empty) - default DSN for every Redis-backed capability (rate limits, login lockout, pending MFA, websocket tickets, event bus).
• STATE_URI / EVENTS_URI / LOCK_URI / STORAGE_URI - fine-grained overrides. STORAGE_URI accepts local:// (default) or s3://bucket/prefix.
• JOBS_ENABLED (default: false) plus JOBS_RUN_IN_PROCESS_WORKER, JOBS_MAX_ATTEMPTS, JOBS_BACKOFF_BASE_SECONDS, JOBS_BACKOFF_MAX_SECONDS, JOBS_LEASE_SECONDS, JOBS_BATCH_SIZE, JOBS_POLL_INTERVAL_SECONDS, JOBS_RETENTION_DAYS - durable job queue tuning.
• APP_ROLE (default: api) - worker runs a dedicated job-worker container from the same image.
• EVENT_LOG_ENABLED / EVENT_LOG_RETENTION_DAYS (defaults: true / 90).
• Docker build arg PYTHON_EXTRAS=distributed - installs the Redis/S3 client libraries. The default image does not include them; if you use REDIS_URL or s3:// storage, build your own image with this extra.

Changed

• SMTP_FROM - Now falls back to SMTP_USERNAME when unset (it was auto-detected by Apprise before). If neither is set, email is disabled with a startup warning. A bare address is sent as Endurain <address>; Name <address> is kept as written.
• SMTP_SECURE - false (no TLS) is now only accepted for a private, loopback or link-local SMTP_HOST, and not together with SMTP_USERNAME/SMTP_PASSWORD. Otherwise email is disabled with a warning. If you relied on unencrypted authenticated SMTP, switch to starttls or ssl.
• SMTP_HOST / SMTP_PORT / SMTP_USERNAME / SMTP_PASSWORD - validated at startup: SMTP_HOST must be a hostname or IP address (no scheme, port or path), SMTP_PORT 1-65535, and username and password must be set together and contain only ASCII characters. An invalid combination disables email with one warning per problem naming the setting (never its value); Endurain still starts.

Removed

• RATE_LIMIT_STORAGE_URI and AUTH_SECURITY_STORAGE_URI - replaced by REDIS_URL / STATE_URI. Endurain refuses to start while either is still set, so replace them with REDIS_URL in your compose file before updating.
• JAEGER_ENABLED, JAEGER_PROTOCOL, JAEGER_HOST, JAEGER_PORT - OpenTelemetry/Jaeger tracing has been removed, along with the Jaeger service in the example compose files.

Other

• New data folders are created at start: activity_files/upload_incoming and activity_files/upload_staging.

Changes from v0.20.0-beta1

• Apprise removed; email now uses the jasil mail provider (#825). Password reset, sign-up verification, admin approval request (one per admin) and sign-up approved emails are published as events (auth.password_reset_requested, auth.sign_up_verification_requested, auth.sign_up_approval_requested, auth.sign_up_approved) and handled by subscribers. With JOBS_ENABLED=true they run as durable jobs with retries and backoff; otherwise delivery is best-effort on the in-process event bus (no retries, lost on restart). The apprise dependency is gone from the image.
• Email is now sent asynchronously. The API no longer reports delivery failure: the 500 Unable to send password reset email response and the "Failed to send verification email" sign-up message are gone. Failures show up in the logs and, with jobs enabled, in the Jobs dashboard. Permanent failures (SMTP 5xx, email not configured, invalid message) are logged and not retried; transient ones (connection, TLS, 4xx) are retried when jobs are enabled.
• Only the newest reset or verification link works. Requesting a new password reset or sign-up verification email invalidates the previous unused link. Before, every unexpired link stayed valid.
• No secrets in events. Event payloads carry user ids only; the reset/verification token is created by the handler and committed before the email is sent, so no token or address is stored in events or jobs.
• Clear startup log for email. One info line when SMTP_HOST is unset, one warning per problem when the SMTP settings are invalid (email disabled), one info line when enabled.
• Email requests from async routes run in the thread pool so they no longer block the event loop (CodeRabbit review on #825).
• Docs: SMTP variable table updated (SMTP_FROM fallback, SMTP_SECURE=false restrictions, retries need JOBS_ENABLED); Apprise removed from the tech stack and feature lists.
• Send a test email from the admin settings. New Email card on Settings > Server: shows whether the SMTP settings are valid (and, when they are not, which settings are at fault, by name only) and a Send test email button. The test is sent synchronously to the admin's own address, never to an arbitrary recipient, and a failure shows the mail server's failure category and SMTP status code (for example "authentication failed (status 535)"). API: GET /server_settings/email (server_settings:read) and POST /server_settings/email/test (server_settings:write, rate limited; 503 when email is not configured, 502 when the mail server refuses the message). The email text is English only for now; other languages fall back to English.
• OpenStreetMap default tile URL (#812). The default and the OpenStreetMap preset now use https://tile.openstreetmap.org/{z}/{x}/{y}.png (no a/b/c subdomains), as OSM's tile usage policy asks; the thumbnail fallback and the frontend fallbacks match. A new migration rewrites a stored legacy OSM URL to the new one; custom tile servers are left unchanged. Fixes #811.
• SSO redirect honors ALLOWED_REDIRECT_SCHEMES in the frontend (#826). The backend already accepted the schemes configured in ALLOWED_REDIRECT_SCHEMES, but the frontend only allowed endurain:, so mobile SSO handoff to other apps (for example Gadgetbridge) never happened. The public server settings now expose the configured schemes and the frontend validates redirects against them; the allow-list is still deny-by-default and unsafe schemes are never accepted as redirect targets.
• Spoofed X-Forwarded-For hops are ignored (#829). The client IP was taken from the leftmost X-Forwarded-For entry, which a client can prepend when the proxy appends to the header, so a forged value could bypass rate limiting and poison session IPs. The header is now read right to left and the first hop that is not a trusted proxy is the client. Check that TRUSTED_PROXIES lists your reverse proxy(ies) when BEHIND_PROXY=true.
• Garmin sleep sync no longer overwrites existing data (#830). For a date without sleep, Garmin returns a sleep record with every field empty. The sync treated it as real data and overwrote any existing sleep row for that date, including rows written by another source, storing None as the Garmin sleep id. Such dates are now skipped, and the sync only updates rows that Garmin wrote.
• Docker compose examples simplified. The multiple-backends example is removed and Redis is no longer part of the default examples: REDIS_URL is commented out in .env.example, and DEPLOYMENT_PROFILE is no longer set there. Add Redis yourself only for a distributed deployment (see Scaling and background jobs).
• Docs: new Nike Run Club export guide in the import docs (#819).
• Docs: new Map tile server section in Advanced setup explaining how the tile URL and the tile server API key are used by browser maps and by backend thumbnails, which providers receive the key in a header (Stadia Maps), and the workaround for providers that expect a different query parameter name (#831).
• Dependency updates. Backend and frontend dependencies bumped to their latest releases and minimum versions raised to match (for example SQLAlchemy 2.0.54, uvicorn 0.53, timezonefinder 9, garminconnect 0.3.16, ruff 0.16.8; Vue 3.5.44, Pinia 4.1, vitest 5.0.5, eslint 10.13). TypeScript stays on 6.0 because TypeScript 7 does not work with vue-tsc yet.
• Tooling and images. uv 0.12.24 (still 0.12.x), astral-sh/setup-uv v10.3.0 pinned by commit hash, and refreshed digests for the node:24-slim and python:3.13-alpine base images.

Breaking API changes

If you only use the Endurain web UI you do not need to do anything. API consumers, note the main route changes (full list in the OpenAPI reference):

Authentication routes (login, refresh, MFA) are unchanged in this release.

General changes:

• Paginated list envelopes. Activity, follower, lap, set, workout-step and stream list endpoints return a paginated envelope with totals instead of bare arrays. The activity feed uses keyset pagination. Empty results are [], never null.
• Nested child resources. Activity child resources (laps, sets, streams, workout steps, media, exercise titles) live under /activities/{id}/....
• 202 Accepted for uploads and provider refresh. File upload and Strava/Garmin refresh now return 202 and run as durable jobs. Uploads support an Idempotency-Key header.
• PATCH for bulk activity edit and activity ETags.
• RFC 9457 problem documents for all error responses (replaces {"detail": ...}), documented in the OpenAPI schema.
• Signed, token-gated URLs for activity media, thumbnails and user profile photos. Tokens expire; fetch fresh URLs from the API rather than caching them.
• Followers API reworked (new model, request/accept notifications, privacy-aware lists; unknown fields in follow decisions are rejected).
• Activity responses use Read schemas; timezone formatting fields were removed from activity and lap schemas. Datetimes are normalised to UTC.
• Thumbnails are now WebP, generated through the event bus and storage provider (#774). Existing PNGs are migrated.

Security

This release includes security fixes, including high-severity issues. Details will be published with v0.20.0.

Thank you to everyone who reported these responsibly.

Backend

• Backend restructured into modules with enforced boundaries; services own transactions and publish events.
• Unified activity ingestion (ActivityCore) with content-hash and dedup_key deduplication, UTC normalisation and idempotent re-imports; activity media deduplicated by content_hash.
• Activity source-file storage, a pluggable file-parser registry and orphan sweeps for files, media and thumbnails.
• Reverse geocoding with location backfill and defer logic for unresolved activities.
• HR zone percentages are no longer stored; they are computed from your current max heart rate when streams are read.
• Thumbnail regeneration with retry scheduling.
• Distributed locking for scheduled jobs; user activity locking during deletion; bulk-deletion events.
• Follower invariants enforced with check constraints; activity exercise-title uniqueness.
• Legacy timezone aliases canonicalised.
• Notifications are created idempotently from source event IDs.
• Week start respected in stats (#910).
• Pruning of event_log, processing_jobs and event_outbox in bounded batches.
• Dependency updates: FastAPI, SQLAlchemy, Alembic, Pydantic, psycopg.

Frontend

• Pending-thumbnail placeholders while map images are generated.
• Feed and search updated for paginated results.
• Gear component dates use calendar dates in the local timezone.
• Vite PWA precache excludes the large login background image.

Docs

• New API reference (OpenAPI) page under Integrate: browse every REST endpoint, parameters and responses without installing anything. It is generated from the backend code at docs build time, so it always matches the release.
• New: scaling and background jobs, reverse proxy, operations, troubleshooting, export/import, error handling, API reference for Garmin/Strava, events and subscribers, activity ingestion, supported types.

Contributors

@ycanerol in #812 and #819