github medizininformatik-initiative/aether v1.5.0
on GitHub

4 hours ago

This release makes the TORCH and flattening steps easier to operate. pipeline start now reads the CapabilityStatement of TORCH and of the flattener before it creates a job, so a wrong URL or wrong credentials stop the run at once. The flattening step detects a flattener that stops answering, no longer retries requests that time out, and logs the sizes of each attribute group. The TORCH extraction progress shows in the terminal, the log and pipeline status. A fix in the ViewDefinition builder stops the flattener from pairing values from different repeating elements. The release also adds a mutation testing gate in CI, and pulls in TORCH v1.0.2 and FHIR-Pseudonymizer v2.38.0.

What's Changed

Breaking Changes

  • pipeline start checks the TORCH CapabilityStatement — When the torch step is enabled, aether sends GET {base_url}/fhir/metadata with the TORCH credentials before it creates a job. A non-2xx status or a body that is not a CapabilityStatement stops the start. HTTP 401 and 403 show as an authentication error. Before, aether sent HEAD to the host and accepted any status. #946 (closes #939, #944)

    Action required — make sure that services.torch.base_url is the server root, not the /fhir path.

  • pipeline start and the flattening step check the flattener — aether sends GET /fhir/metadata to the flattener and requires a CapabilityStatement that declares the $run operation. A service that is not a flattener fails the check without a retry. #953 (closes #950)

Features

  • Detect a flattener that stops answering — During a $run request, a watchdog probes /fhir/metadata every 30 s. After 3 failed probes in sequence, the step fails with "flattener stopped answering", about 2 minutes after the flattener stops. Before, the step waited for the full timeout (30 minutes by default). #953 (closes #950)
  • Log the resource and batch sizes of each flattening group — An INFO line Flattening group summary for each group, and a WARN line for a resource that is larger than the batch limit. #956 (closes #951)
  • Make the TORCH request timeout configurable — New key services.torch.request_timeout (default PT1M). Before, TORCH had only 10 s to accept an extraction request. #941 (closes #940)
  • Show the TORCH batch progress — aether reads the torch-job-progress extension from the TORCH Task API and shows it in the terminal and the log. #832 (closes #839)
  • Persist the TORCH progress and show it in pipeline status — The progress is available from a second terminal while the extraction runs. #842 (closes #840)
  • Write rejected request bodies to disk — New option services.send.dump_failed_requests. When the transfer server refuses a request, the send step writes the request body and the answer to jobs/<job-id>/send/failed/. #833 (closes #820)

Bug Fixes

  • Use one forEach loop for attributes with a common ancestor — Attributes below one repeating element got separate loops, and the flattener paired values from different elements (for example, the code of ingredient A with the strength of ingredient B). #964 (closes #960)
  • Stop retrying flattener requests that time out — The flattener continues to work on an abandoned request, so each retry added load. DIMP, validation and TORCH keep their retry behavior. #947 (closes #942)
  • Summarize lookup file warnings in one short log line — One WARN line with the number of findings per code; --verbose shows the full list. #948 (closes #943)
  • Stop the ViewDefinition builder from writing into the lookup table — Removes a data race between two builds on one lookup table. #933 (closes #904)
  • Keep the http.DefaultTransport defaults in BuildTLSTransport — A custom CA certificate no longer removes the proxy support, HTTP/2, timeouts and connection-pool limits. #849 (closes #813)
  • Select the size unit after the value is rounded — One byte less than 1 GB shows as 1.00 GB, not 1024.00 MB. #888 (closes #887)
  • Stop the e2e scripts from failing when grep closes the pipe early #931 (closes #909)
  • Make the default mutation run test the internal directory #932 (closes #930)

Internal & CI

  • Mutation testing — make targets mutation and mutation-diff #886 (closes #882), a CI gate for pull requests with an efficacy threshold of 95 % #938 (closes #870), and tests that kill the surviving mutants #883 #885 #894 #895 #896 #897 #898 #899 #905 #908 #913 #914 #917 #918 #919 #920 #925 #926 #927 #935
  • Mock TORCH server that reports batch progress #841 (closes #838)
  • Split the pipeline start handler and add tests #945 (closes #944)
  • Share one URL check between the service validators #952 (closes #949)
  • Split the flattening models file by topic #929 (closes #900)
  • One helper for atomic file writes #847 (closes #834)
  • Remove untestable checks from the TORCH download #890 (closes #889)
  • Record a progress sample at a given time #876 (closes #851)
  • Remove the fixed waits from the slow unit tests #848 (closes #846)
  • Track the shared Claude Code config and ignore local state #831 (closes #829)

Dependency Updates

Full Changelog

Check out latest releases or
releases around medizininformatik-initiative/aether v1.5.0

Don't miss a new aether release

NewReleases is sending notifications on new releases.

Get notifications