Freika/dawarich 1.17.0 on GitHub

Dawarich gets lighter and faster in this release. On a test account with a million points over three years, compared with 1.16.0:

It needs about half the memory at idle: 230 MiB instead of 480 MiB.
Long trips open in a fraction of the time. A 300-day trip no longer downloads every recorded point: the page and its map need about 1 MB instead of 11 MB of page and 190 MB of point data, and the trips list opens in 0.1 s instead of 2 s.
Paging through the points page and the points API is up to four times faster, and repeat visits are answered almost instantly.
While new points keep arriving, map tiles for other months are no longer downloaded again.
A 200,000-point Google Records import finishes in 132 s instead of 175 s, and big imports no longer need memory in proportion to the file size.

Upgrading a self-hosted instance

Background jobs now run inside the dawarich_app container, in the same process as the web server, so the separate dawarich_sidekiq container is no longer needed. Jobs waiting in Redis are kept and picked up after the upgrade.

With your old docker-compose.yml still in place, stop and remove the worker so it can finish or return its running jobs:
```
docker compose stop -t 60 dawarich_sidekiq
docker compose rm -f dawarich_sidekiq
```
Update the Compose file in one of two ways:
- Replace it with the new docker-compose.yml, keeping your own environment values, volumes, Redis and database settings.
- Or edit your own file: delete the dawarich_sidekiq service, add stop_grace_period: 60s to dawarich_app, and under its existing environment: replace WEB_CONCURRENCY and add two variables:
```
SIDEKIQ_EMBEDDED: 'true'
WEB_CONCURRENCY: '0'
BACKGROUND_PROCESSING_CONCURRENCY: 2
```
  WEB_CONCURRENCY must be 0: every extra web process would start its own set of job threads.
Move any setting you had only on dawarich_sidekiq, such as SMTP, OIDC or reverse geocoding, to dawarich_app. If your .env sets BACKGROUND_PROCESSING_CONCURRENCY=10 from the old .env.example, remove it or set it to 2, and remove RAILS_MAX_THREADS if it is set. If it sets APP_CPU_LIMIT=0.50 from the old .env.example, raise it to 1.00.
Start the stack and check that three containers are running: dawarich_db, dawarich_redis and dawarich_app.
```
docker compose pull
docker compose up -d
docker compose ps
```

Don't start the old worker again next to the new setup. Larger instances, or ones with frequent big imports, can keep jobs in a separate container instead; see Background jobs.

Added

Import your check-in history from a Foursquare Swarm data export. Upload the whole export archive or its checkins*.json files: every check-in becomes a point, and every check-in at a venue becomes a visit at a place named after that venue. A visit lasts until your next check-in, at most two hours. The other files in the archive (tips, photos, ratings) are skipped.

Changed

The app container in the default docker/docker-compose.yml is limited to one CPU core instead of half a core, since it also runs background jobs. Set APP_CPU_LIMIT to change it.
The Docker image now ships Node.js 22 LTS from the official Node.js build instead of Debian's Node.js 20 package, which no longer receives upstream security fixes. Node.js runs the server-side poster renderer. #3888
The points page and the points API page through large accounts much faster.
Reverse geocoding gets through a backlog faster: it uses as many background threads as your provider's rate limit allows and keeps its connection to the provider open. If you use Geoapify, LocationIQ or the public Nominatim, set a requests-per-second limit in the geocoding settings or REVERSE_GEOCODING_RPS, since Dawarich does not limit those providers on its own.
Visit detection and track generation need less memory and fewer database queries, and the place-name refresh skips months where nothing changed.

Fixed

Polish counts use the right plural form, for example "3 kraje" and "5 krajów" instead of "3 krajów" and "5 kraju". This also fixes relative times such as "około 3 godziny" and labels on the map.
Trips with long recordings open quickly: the trips list, trip page, trip form and shared trip page draw routes from map tiles instead of sending every recorded coordinate, and the trip page loads points only when you open Replay, the poster or the video studio (#3867, #3783)
Overnight stays where the phone goes quiet, for example arriving home in the evening and leaving in the morning, are now suggested as one visit across the silence instead of a few minutes after arrival. Choose Re-run detection on full history on the Visits page to fix past stays.
Points synced from TeslaMate now get suggested visits; choose Re-run detection on full history to cover earlier drives. On Dawarich Cloud, trial accounts are now included in the nightly visit run.
Re-running detection on full history no longer skips months with more than 100,000 points.
Opening a calculated trip whose countries could not be determined, for example without a reverse geocoder, no longer recalculates its path, distance and countries on every view; TREK sync no longer does either. If your points get countries later, press Recalculate on the trip page (#3868)
"Where you spent" on Insights counts days and year boundaries in your timezone, so a stay across New Year no longer shows the first days of the year at the end of the ribbon (#3869)
Countries you only flew over no longer count toward trips, monthly and yearly digests, or days per country on Insights. (#2711)
Track elevation gain and loss ignore small GPS altitude fluctuations. (#3458)
Photo markers on a public shared trip are spread across the whole trip, so later days still appear within the 100-marker limit. (#3653)
When a tracker's motion data is marked authoritative, the transportation mode it reports is kept during stops. (#3668)
Tracks are colored by speed again, and the speed color scale can be edited in the map settings. (#3681)
The heatmap is weighted by time spent at a place instead of how often points were recorded there. (#3597)
POST /api/v1/visits/batch accepts the same optional address as POST /api/v1/visits, and a visit sent with an address that has no street or city still gets its place address looked up.
A visit whose address is not an object is created without an address instead of failing.
The year digest counts time by country and travel patterns in your timezone, so a day around New Year is no longer added to the wrong year or split across two days. Choose Recalculate tracks & stats to refresh digests of past years.
Drives reported with a speed of 0 (OwnTracks, background Android) are no longer classified as stationary. Choose Recalculate tracks & stats to reclassify existing tracks (#3887)
Real-time tracks now split only on the time gap, like imported history and the map, so sparsely sampled driving points are no longer left without a track. Choose Recalculate tracks & stats to fill gaps recorded before this release (#3638)
End tracks when a device stays within 100 m for longer than the minutes-between-routes setting, even if it keeps reporting locations, so overnight stops can be suggested as visits again; regenerate existing tracks to apply the fix (#3356)
Continent achievements count the same countries as their explored regions, including time spent at points without a country tag; stored progress is recalculated and may unlock countries (#3784)
Photo clusters and their counts stay visible above tracks, heatmaps and points on the map (#3770)
The trip map refreshes after the live connection reconnects, even when the path was calculated before it connected (#3736)
GeoJSON and account-archive imports keep very precise coordinates near zero; such a longitude, for example -0.12345678901234567, was stored as -128.
Import archives with many photos or other unsupported files no longer fail with "Archive too large" or "Too many files".
Imports for accounts being deleted are skipped instead of failing, and account deletion waits for running imports to finish.
Raw geocoding responses, which contain addresses and coordinates, are no longer written to the log.
The visits API answers with a bad-request error (Invalid time range, previously Invalid date format) for malformed or reversed date ranges, and for area queries that send only one of start_at and end_at. Creating a visit rejects impossible dates such as 30 February instead of moving them into March.

💙 This release is supported by Steven B., James Manolios, chenrik, aldumil, derpderpington, Chippie, dint, jhalpern, Lex Fradski, Schlufo, cyberswan.at, craftyklaus, Andre, hogenf, naraxius, Embrace, martin4861, Alex, evetters, GregF, Jon Coffee, Lukas, Robbie G, Kilian, Hans G, Chris, tabaha, Andre, Michael C, Chris, Jonah B., Dante, daallex, Tanner A., Milien M., Mathias, Travis S., Matthew F., Johnathan D., bleibdirtroy, no1etal, dixet, Bailey J., Alex D., Benjamin M., Daniel A., Florian, BeeHappy, ChemistryDachshund, OrangeWindies, Michelangelo V., Edward, Erazem Z., Denis Abt, Tony G., Lomky, Osamu Y., Linus T., Christian C., Sebastian, Jan K., Nathan T., Max G., Karol B., Konstantin, Johannes H., t3hero, g3lin, Tim, Philipp M., Brand K., Pablo F. M., Jon H., fkB, Mikael, Dániel A., Hai_Tsung, ShooTex, GreenTentacle, PinkahPandah, David, Ken, Denna G., Andre, Hendl91, Jon H., MrRed, Arkadiusz Z., Jake, Bator T., Alex J. H., eps-dev, twiggu, Dragan V., Jerome G., Sheya B., Gustav B., higgs01, Frank F., F. J. Kruz, Jon D., Alexander J. H., Daniel D., Ricardo T., Christof Z., Narrator, JohnSmith21, smartbert, A.M., Matthew Z., Dror T., Jonathan K., Kyle C., Mario B., Albin H., Phillip S., Bernd, Tom, Nick O., greeting7416, Dick, Brian R., Markus, TheSlimShady, Andrew M., Mason B., Chance, GoogilyBoogily, Dragon, Oliver, HexitNiels, Michael G., Pablo A., Leo L., Daniel D., DSKun, Jonathan K., Erik K., Tiebolt v. d. L., Steven, Mathias, Sam B., Manuel F., K8BCO, Samuel D., duckduck, David, heydabop, Riccardo N., Tim. S., Jonas, Kenji I. Pascal M., Raymond W., Julien, Roman, Mario B., Thomas G., Gabriel, Ivan A., Aram B., Dan551 and Andreas R. on Patreon and KoFi 💙

What's Changed

fix: exclude flown-over countries from trips, digests and residency by @Freika in #3860
fix: ignore GPS altitude noise in track elevation gain and loss by @Freika in #3861
fix: spread shared trip photo markers across the whole trip by @Freika in #3862
fix: honor authoritative tracker motion mode during classification by @Freika in #3863
fix: restore speed coloring on the Tracks map layer by @Freika in #3864
fix: weight heatmap by time spent instead of point count by @Freika in #3865
Accept addresses in visit batches and ignore malformed ones by @Freika in #3870
1.17.0 by @Freika in #3894

Full Changelog: 1.16.0...1.17.0