github medizininformatik-initiative/dataportal v7.0.0

one hour ago

[7.0.0] - 2026-11-18

Overview

This release makes the data use project (DUP) pipeline of the data-node production ready. TORCH and aether reached their first stable versions, the DIMP step supports per-project pseudonymisation, and the data selection uses the same search as the cohort selection. No services were added or removed.

Changes in this repository:

  • Example DUP project with per-project dynamic pseudonymisation - see
  • DIMP DUP configs mask required references with a data-absent-reason instead of pruning them, keep the re-pseudonymized patient and encounter identifiers and redact all other identifiers
  • Consent documentation and an example consent CRTDL - see
  • Combined transfer load script for the send step, test data switched to the MII test data, data-portal env defaults use the context path setup (backend under /backend)
  • Automated integration tests in CI - see
    • data-portal: backend API contracts, ontology checks, feasibility counts against golden values, availability-updater and browser tests of the UI
    • data-node: TORCH API, extraction and consent, aether CLI, DIMP, fhir-flattener, Blaze auth and the full DUP pipeline for every DSE profile
  • Fixed the default AVAILABILITY_REPORT_SERVER_BASE_URL of the availability-updater (fhir-sever → fhir-server)
  • Fixed the Keycloak database healthchecks of the data-portal and the data-node, which ran pg_isready with an empty user and database
  • Fixed the flare certs mount, which was mounted read-write instead of read-only

Breaking changes

  • postgres: The dataportal-backend database (dataportal-postgres), the data-portal keycloak database (auth-db) and the data-node keycloak database (fhir-server-keycloak-db) are upgraded from Postgres 17 to 18. Postgres 18 cannot start on Postgres 17 data, and the volume mount moved from /var/lib/postgresql/data to /var/lib/postgresql. Existing installations have to migrate their data. The commands below run from the repository root with the default compose project dataportal and the default database users.
    1. Before updating, dump each database with the old version running:

      • backend: docker compose -p dataportal -f data-portal/backend/docker-compose.yml exec dataportal-postgres pg_dumpall -U dataportaluser > backend-dump.sql
      • data-portal keycloak: docker compose -p dataportal -f data-portal/keycloak/docker-compose.yml exec auth-db pg_dumpall -U keycloakdbuser > keycloak-dump.sql
      • data-node keycloak: docker compose -p dataportal -f data-node/fhir-server/keycloak.docker-compose.yml exec fhir-server-keycloak-db pg_dumpall -U keycloakdbuser > node-keycloak-dump.sql
    2. Stop the stacks and remove the old volumes (dataportal-postgres-data, dataportal_dataportal-auth-db, dataportal_fhir-server-keycloak-db).

    3. Update the repository and start only the databases, for example docker compose -p dataportal -f data-portal/keycloak/docker-compose.yml up -d auth-db. Do not start Keycloak or the backend yet. They would create a new schema in the empty database, and the restore would then fail.

    4. Restore each dump:

      • backend: docker compose -p dataportal -f data-portal/backend/docker-compose.yml exec -T dataportal-postgres psql -U dataportaluser -d postgres < backend-dump.sql
      • data-portal keycloak: docker compose -p dataportal -f data-portal/keycloak/docker-compose.yml exec -T auth-db psql -U keycloakdbuser -d postgres < keycloak-dump.sql
      • data-node keycloak: docker compose -p dataportal -f data-node/fhir-server/keycloak.docker-compose.yml exec -T fhir-server-keycloak-db psql -U keycloakdbuser -d postgres < node-keycloak-dump.sql

      The errors role ... already exists and database ... already exists are expected.

    5. Start the stacks as usual.

  • ontology: Ontology 5.x changes the Elasticsearch artifact. Reinstall the index once with DATAPORTAL_ES_INIT_FORCE_REINSTALL="true".
  • TORCH: Consent results differ from 6.0.0 - see
    • Only .8 and .6 are evaluated and both must be in the CRTDL. Combined consent keys (e.g. yes-yes-no-yes) are no longer supported.
    • The consent start moves only to the start of an inpatient stay that began at most TORCH_ENCOUNTER_SHIFT_MAX_DAYS (default 7) days before.
    • Patient.identifier is no longer a standard attribute. Add it to the CRTDL if you need it, e.g. for re-pseudonymisation.
  • fhir-pseudonymizer v3: Keys must be at least 32 bytes (openssl rand -hex 32). URL-like values that contain ~ or - are hashed differently, so such pseudonyms in 6.x extractions cannot be linked to 7.0.0 extractions.
  • aether:
    • Call it as aether pipeline start <aether.yaml> <crtdl.json>. The --config flag is gone.
    • The DIMP output directory is now dimp (was pseudonymized).
    • Service URLs are server roots without /fhir.
    • services.dimp.experimental_v3 is replaced by services.dimp.anonymization_config.
    • The first Ctrl+C asks for a confirmation. Scripts should send SIGTERM or use aether job stop.
  • Blaze 1.13.0:
    • The image no longer contains curl, so healthchecks must use wget.
    • The frontend ignores ORIGIN and uses HOST_HEADER and PROTOCOL_HEADER. Users have to sign in again, and the reverse proxy must accept request headers of about 6 KB.
  • mii-fhir-validator: Copy the new IG_PARAMS from .env.default into your .env and update the terminology in the local Blaze - see
  • FLARE: Unsupported FHIR search parameters now fail a query instead of being ignored (FLARE_FHIR_STRICT_HANDLING, default true).
  • dataportal-backend: The REST API moved to /api/v6. External API clients have to update their paths.
  • TORCH 1.1.0: Compartment resources without a patient reference are excluded as PATIENT_NOT_FOUND, resources linked to several patients as MULTIPLE_PATIENTS. job-summary.json reports both counts.
  • vfps: The vfps database is upgraded from Postgres 17.5 to 18.7. It has no named volume, so the updated container starts with an empty database and existing pseudonyms are lost. If you need them, dump the database before updating with docker compose -p dataportal -f data-node/fhir-pseudonymizer/docker-compose.vfps.yml exec postgresql pg_dumpall -U postgres > vfps-dump.sql. After the update start only the database with docker compose -p dataportal -f data-node/fhir-pseudonymizer/docker-compose.vfps.yml up -d postgresql, restore with docker compose -p dataportal -f data-node/fhir-pseudonymizer/docker-compose.vfps.yml exec -T postgresql psql -U postgres -d postgres < vfps-dump.sql and then start the data node.

Security

  • Java4Shell (CVE-2026-114938): All JVM services run with -XX:-UseBASE64Intrinsics. Elasticsearch sets it in ES_JAVA_OPTS, Keycloak in JAVA_OPTS_APPEND, the data-node services (Blaze, flare, TORCH, FDE, fhir-flattener, mii-fhir-validator) in JDK_JAVA_OPTIONS. The dataportal-backend image sets the flag itself. If you override these variables in your own setup, add the flag there.
  • nginx HTTP/2 (CVE-2026-123633, CVE-2026-123007): HTTP/2 is disabled in the data-portal and data-node proxies, because no fixed nginx-unprivileged image is published yet. Clients fall back to HTTP/1.1. #226 tracks the image update and re-enabling HTTP/2.

Ontology

This release is based on ontology version v5.2.2 (was v4.0.0).

  • New modules: Onkologie and MolGen 2026
  • New attributes: interpretation for laboratory values, status for encounters
  • Flattening lookup for aether (flattening.zip) is now part of the ontology release
  • New Elasticsearch index for profile details, with German umlaut handling

Updates since last Major

The versions of all components per release are listed in the compatibility matrix.

Data Portal

Component From To
dataportal-ui v6.8.0 v7.3.4
dataportal-backend v8.6.0 v9.5.2
availability-updater v0.3.0 v0.4.2
dataportal-es-init 2.0.0 3.1.0
Elasticsearch 9.2.4 9.5.6
Keycloak 26.3.1 26.8.2
Postgres (backend and Keycloak) 17 18.7
nginx (proxy) 1.27 1.31.7

Data Node

Component From To
FHIR Server - Blaze v1.6.2 v1.13.0
terminology server (blaze) v1.6.2 v1.13.0
TORCH v1.0.0-alpha.18 v1.1.0
FLARE v2.7.0 v2.9.0
FDE v1.3.3 v1.3.3
aether v0.6.0 v1.6.0
fhir-pseudonymizer v2.23.0 v3.1.2
vfps v1.3.6 v1.28.0
mii-fhir-validator 0.0.1-alpha.5 0.0.1-alpha.9
Postgres (vfps) 17.5 18.7
Keycloak (FHIR server auth) 26.3.1 26.8.2
Postgres (Keycloak) 17 18.7
nginx (rev-proxy) 1.27 1.31.7
fhir-flattener 0.1.0-alpha.7 0.1.0-alpha.10

The full list of changes is in the release notes of each component, linked in the table.

Highlights

  • dataportal-ui: Angular 19. Elasticsearch-based search replaces the tree in the data selection. Criteria are edited on their own page. Shortcuts lead to missing filters and unlinked references. CSV download of the data selection. Collapsible criteria tiles, layout for lower screen resolutions, more robust result polling and linux/arm64 images.
  • dataportal-backend: Spring Boot 4. Feature search with field descriptions and pinned profiles. sq2cql replaced by cctb-cql. The CRTDL upgrade removes filters the ontology no longer supports. Startup check that the Elasticsearch index matches the ontology version (ELASTIC_SEARCH_ONTOLOGY_VERSION_CHECK_MODE, default warn).
  • availability-updater: Reads the ontology 5.x layout and uses less memory. OAuth2 and basic auth for the ontology download.
  • TORCH: First stable release. Task API to submit, pause, cancel and delete jobs and to follow their progress. Consent calculation follows the MII consent specification and reports conflicts. Extraction-exclusion list and Prometheus metrics. Masks required elements with data-absent-reason stubs and reports resources that still violate required cardinality as job issues.
  • aether: First stable release. Checks the CRTDL, the flattening lookup and the connected services before a job starts. New commands aether job stop and aether send. Re-attaches to running TORCH extractions.
  • fhir-pseudonymizer: Per-project anonymization config on $de-identify and per-context key derivation.
  • mii-fhir-validator: 2026 MII KDS packages, HL7 validator 6.9.11.
  • Blaze: Faster small transactions and JSON parsing. Optional iss and aud checks for JWTs. Security fixes.
  • fhir-flattener: RFC 4180 CSV, lower memory use.
  • FLARE: Strict handling of search parameters. FDE: no changes.

Don't miss a new dataportal release

NewReleases is sending notifications on new releases.