medizininformatik-initiative/aether v1.6.0

on GitHub

This release adds two commands. aether job stop stops a job, also when a different process runs it, and cancels its TORCH extraction. aether send sends the CSV files of a directory to the DSF transfer server or to S3, without a CRTDL and without a pipeline. The first Ctrl+C now asks for a confirmation, and the second Ctrl+C stops the job. The send logic moves into its own package internal/send, and five long functions are split to reduce their CRAP score. The release updates Blaze to v1.13.0, vfps to v1.28.0 and the aws-sdk-go-v2 monorepo.

What's Changed

Breaking Changes

• The first Ctrl+C asks for a confirmation — The first SIGINT prints "Press Ctrl+C again to quit", and the job continues. The second SIGINT stops the job with exit code 130. SIGTERM stops the job at once with exit code 143, as before. If a script or a supervisor stops aether with one SIGINT, send SIGTERM or use aether job stop. The stop does not cancel the TORCH extraction, and pipeline continue re-attaches to it. #1026 (closes #1020)

Features

• aether job stop <config> <job-id> — The command stops a job, also when a different process runs it. It cancels the TORCH extraction and records the job as stopped. Then pipeline continue submits a new extraction. The command sends the stop request through a marker file in the job directory, not through a signal, so it also works on Windows. #1034 (closes #1033)
• aether send <config> <dir> — The command sends all .csv files of a directory and its subdirectories through transfer_load or s3_upload. It reads the files in place and makes no copy of the patient data. It records the name, size and SHA-256 of each file in jobs/<id>/source.json. Resume with --job <id>. If a file changed after the start, the resume stops with an error. A configuration with only services.send and jobs_dir is sufficient. Send jobs have no state.json, so aether job list does not show them. #1069 (closes #1065)

Internal & CI

• Move the send logic into the package internal/send — The pipeline send step and aether send use the same send.Send function. The pipeline sends the same data with the same zip entry names and S3 keys. #1066 (closes #1064)
• Split five long functions to reduce their CRAP score — ProjectConfig.Validate, PipelineJob.Validate, ValidateImportSource, importStep.Run and CreateJob. The behavior and the error messages do not change. #1068 (closes #1060)
• Fix the flaky test of the flattener watchdog — The test allows the one probe that is in flight when Flatten returns. #1052 (closes #1050)
• Run the make lint hook of Claude Code before git commit — The hook matcher did not match a tool name, so the hook did not run. #1070

Dependency Updates

• Go: aws-sdk-go-v2 monorepo #1055 #1073 #1078 #1082 #1090, github.com/klauspost/compress v1.20.4 #1088, golang.org/x/text v0.44.0 #1091, golang.org/x/sync v0.25.0 #1089, lock file maintenance #1058
• Services: Blaze v1.13.0 #1053, blazectl v1.6.0 #1063, vfps v1.28.0 #1079 #1086, SeaweedFS v4.53 #1062 #1085, postgres 18.6 digest #1057 #1075 #1080, debian:bookworm-slim digest #1074
• Tooling: Vale v3.26.0 #1056, step-security/harden-runner v2.22.4 #1081, sigstore/cosign-installer v4.1.3 #1072, github/codeql-action #1067 #1071, docker/login-action #1076, softprops/action-gh-release #1077, fhir-ontology-generator v5.2.0 #1087
• Docs site: Vue v3.5.45 #1084, Vite v8.3.6 #1083

Full Changelog