ulsklyc/yuvomi v2.89.0

on GitHub

Foundation

• A readiness endpoint /readyz next to /health. /health only says that the process
  answers, and it stays 200 during a restore so that Docker does not restart the container in
  the middle of one. /readyz says whether the instance can serve right now: 200 with
  {"status":"ready"}, or 503 with {"status":"not_ready","reason":"..."} while a restore is
  running, the database connection is closed, the database is older than this version expects or
  the data folder refuses writes. It needs no login, tells nothing else about the instance and
  does not write to the database. /health and every shipped healthcheck stay as they are; the
  new path is for a load balancer or an uptime monitor.
• Unexpected errors are answered and logged in one place. The central error handler now
  keeps the status of a client error that is marked as safe to show; until now every error
  that reached it was a 500. Its sentence and reason reach the client only from a
  HttpError (server/utils/http-error.js), whose text we write ourselves; a 4xx of the HTTP
  libraries gets a fixed sentence such as "The request was refused (Bad Request).", because
  body-parser wraps foreign text (zlib) that way. Requests the HTTP layer refuses now get their
  4xx instead of a 500: a corrupt compressed body, an aborted upload or a body shorter than its
  Content-Length 400, a form nested too deep 400, too many form fields 413, an unsupported
  Content-Encoding or charset 415. A path parameter that does not decode (%ZZ) is a 400.
  Everything else stays a 500 with the fixed sentence, and the raw text stays in the log. The
  log line names the method, the route pattern and the status instead of the path, and the
  raw segment of an undecodable parameter is no longer written, so a calendar feed token in the
  URL no longer lands in the log. The cause of an error is logged with it.
• The status of the last backup survives a restart. After a restart the backup page said
  "never" until the next scheduled run, because the last attempt lived in memory only. The time of
  the last attempt, the time of the last success and the reason of a failed attempt are now kept
  in the database, without a schema change, and a restore keeps the values of the installation it
  runs on rather than those inside the restored file. GET /api/v1/backup/status gains
  scheduler.lastSuccess.
• A restore drill checks that a release database restores and migrates on this version.
  scripts/restore-drill.mjs restores a database through the same restore command an operator
  uses, migrates it to the current schema and checks its integrity, foreign keys, schema and row
  counts before it starts the server on it and reads every module. The rollback probe that
  tries the way back before a foundation release moved into the repository as
  scripts/rollback-probe/. Nothing changes for an installation.
• Releases that change the database schema carry a frozen demo database. A new workflow
  attaches yuvomi-<version>-corpus.db and its .sha256 to every release whose schema differs
  from the last one with such a file, and to every foundation release. It is the demo seed of
  that release, encrypted with a public test key, and the restore drill migrates each of them
  to the current version. Nothing changes for an installation.
• Every day and in every pull request, all of those databases are restored and migrated.
  A new workflow downloads each release's frozen database, restores it the way an operator
  would, migrates it to the code under test and starts the server on it. A migration that
  breaks an older database now fails in CI, and the run is also red when the newest release
  changed the schema but has no frozen database. Nothing changes for an installation.
• Lint, layer and dead-code checks run on every pull request. A new lint job runs
  oxlint, dependency-cruiser, knip and zizmor and compares their findings with
  lint-baseline.json, one count per file and rule that only sinks: a new finding or a rising
  count fails, as with the type baseline. An assignment to innerHTML and an await inside a
  database transaction have no baseline and fail at the first finding. Contributors run
  npm run lint before a push; CONTRIBUTING.md lists the rules. Nothing changes for an
  installation.
• Tests that read source code as text are listed and counted.
  test/text-reader-inventory.json names every suite that reads a file under public/ or
  server/ as text, with the files it reads and what becomes of it. A new suite of that kind
  without an entry fails npm run test:text-reader-inventory, and the number of checks that
  hold a spelling instead of a behaviour only sinks. docs/test-suites.md has the rules. Nothing
  changes for an installation.
• CI runs the test suites side by side and takes about ten minutes instead of thirty.
  The Tests job now runs npm run test-parallel with four jobs instead of npm test. A red
  suite no longer hides the ones after it: the job log ends with every red suite and its log,
  and when one is red, the logs are attached to the run. Contributors run npm run test-parallel before a push.
  A new push to a pull request cancels the run it replaces.
• The way back to 2.88.1 was tried before the tag. Image 2.88.1 and this build ran in turn
  on the same demo database, in production mode with a generated encryption key. Neither start
  applied a migration, every module answered the same through the API before and after, and
  the service worker switched from 2.88.1 to 2.89.0 and back without an error screen or a
  failed request, with the session staying signed in. A backup made on 2.89.0 restored on
  2.88.1 through the admin restore, and 2.88.1 started on a database that carries the new
  backup status without a word about it. To go back, pin the image tag 2.88.1.

Added

• Admins can ask the server what it runs on. GET /api/v1/system/diagnostics answers with
  the version and build revision, the Node.js and SQLite versions, platform, schema version and
  the time of the last migration, the size of the database and its write-ahead log, free space on
  the data volume, uptime, the status of the last backup with the next scheduled run, where the
  database key comes from, readiness and the number of installed modules. It never carries a key,
  a path, a hostname, the value of an environment variable or anything about a member, and it is
  refused for members and for signed-in devices. GET /api/v1/backup/status gains
  scheduler.nextRun. The settings page does not show any of this yet.

Changed

• The container image of a release is now built from its tag alone. Until now a release
  build reused layers from the build cache that builds of main fill, the dependency install
  and the app build included. A tagged build now ignores that cache and writes none; only the
  rolling main image keeps it. Nothing changes for an installation.
• A failed backup says why, with a reason instead of the text of the error (from #2525,
  the reason on the last error proposed by @nomi25home in #2540). scheduler.lastError in
  GET /api/v1/backup/status now always carries reason, one of backup_dir_not_writable,
  disk_full, backup_verify_failed and unknown, and its message is the fixed sentence of
  that reason. Until now message was the raw text of the error, with the absolute path of the
  backup folder in it; that text is in the server log only now, where it still names the folder
  and what to check. POST /api/v1/backup/trigger answers a failed run as before with 200 and
  success: false, and carries the same reason. The backup page does not show the reason yet.

Fixed

• The demo seed finds the database key the server stored itself. scripts/seed-demo.js --db <path> now reads DB_ENCRYPTION_KEY_FILE and the key a first start keeps under
  .secrets/ next to the database, in the same order as the server. Until now it stopped on an
  encrypted database unless the key was in the environment or in .env.