XNAT 1.10.2 Release Notes

XNAT 1.10.2 is a maintenance release focused on DICOM handling. It includes security fixes, and we recommend it for all 1.10.x sites. It requires no change to your Java, Apache Tomcat, or PostgreSQL versions from 1.10.1, makes no database schema changes, and bundles DicomEdit 6.10.0. Sites that deploy the XNAT WAR into their own Tomcat, or build their own container image instead of using the XNAT image, need to install one new native library. Some changes affect existing behavior, scripts and plugins, so be sure to review the Before You Upgrade section.

Highlights

• Snapshots and thumbnails for JPEG-compressed scans, including JPEG Baseline, and for segmentations, mixed single- and multi-frame series, and scans in the secondary DICOM catalog.
• A new, more efficient pixel redaction engine for alterPixels that keeps lossless transfer syntaxes and uses the requested fill value. We recommend verifying existing scripts that use alterPixels with DicomEdit 6.10.0.
• Anonymize and redact very large DICOM objects (e.g., Pixel Data over 2 GiB), with pixel data streamed from disk instead of held in memory.
• Delete a direct-archive session's files along with its record, and give non-admin users with All Data Access read-only access to the whole prearchive.
• Use DICOM elements that come after Pixel Data, including private tags in groups 0x8000 and above, in routing rules, DICOM mappings, and the DICOM dump.
• A fix for a deadlock that could stop a node.
• Stability improvements and security hardening.

Sites that deploy the XNAT WAR into their own Tomcat, or build their own container image instead of using the XNAT image, should install one new native library when upgrading. See Before You Upgrade.

Before You Upgrade

OpenCV native library

XNAT now decodes JPEG-family pixel data (JPEG, JPEG-LS, JPEG Lossless and JPEG 2000) through dcm4che's OpenCV codecs. That's what lets it generate snapshots of these studies and redact their pixels.

• XNAT's container image: the library is already included.
• Your own Tomcat or your own image: install the library on the JVM's java.library.path before upgrading, following Installing the OpenCV native library..

Pixel redaction output

Pixel redaction (alterPixels) output changes relative to 1.10.1 in two ways:

• Uncompressed and lossless objects keep their transfer syntax instead of becoming Explicit VR Little Endian.
• Scripts use the requested fill value instead of 0.

See DicomEdit 6.10.0 for more details.

Disk space

• Anonymization, and redaction of uncompressed data, now handle objects over 2 GiB, and both write working files to disk. As before, anonymization writes its working copy to an anon_backup folder in the JVM's temporary directory java.io.tmpdir, which in the XNAT container image is /usr/local/tomcat/temp.
  • Redaction scratch files go to java.io.tmpdir too, unless the JVM system property dicom.pixeledit.scratch.dir names another directory.
  • Make sure these locations have room for several of your largest objects at once and are readable only by the XNAT user, since the files hold image data that may not yet be fully de-identified.
• Zip and tar files that XNAT extracts on upload, into a resource (extract=true), the user cache, the automation-based uploader, or a XAR import, are now written to the cache path first, in UploadBuffer, and deleted after extraction. Make sure the cache volume has room for the largest of these uploads. DICOM imports, including the compressed uploader, are unchanged.

Site configuration

• If you override Login.vm, Register.vm, DefaultTop.vm, RegisterExternalLogin.vm or PrearchiveDetails.vm, merge in the 1.10.2 changes.
• Remove any dcm4che jars you added to ${xnat.home}/plugins yourself; an older dcm4che-imageio-opencv there breaks compressed snapshots.

Plugins

• Some APIs and libraries changed or were removed, including DicomEdit's pixelmed-based redaction classes, the direct-archive service interfaces, and Springfox. See Plugin Developer Notes.

Anonymize and Redact Very Large DICOM Objects

• XNAT-8737, XNAT-8614: Anonymization now streams pixel data from disk instead of loading it into memory, wherever it runs: site-wide on import, project on archive, and re-anonymization on relabel, project move and label edits. Heap use no longer grows with object size, so sites with anonymization enabled need less heap for large objects and many concurrent imports.
• Objects whose Pixel Data exceeds 2 GiB can now be anonymized, and redacted. A compressed object can be redacted only with the OpenCV native library installed (and only if it decodes to have Pixel Data < 4 GiB). See the redaction limits under DicomEdit 6.10.0, and Disk space under Before You Upgrade.
• Sites that turned on Native DICOM Pre-Compression (Administer > Site Administration > Session Upload, Import & Anonymization) to work around this no longer need it for anonymization. If left on, it still compresses large files, but only when the OpenCV library is installed.

Use DICOM Elements After Pixel Data

• XNAT-7932, XNAT-7933: Custom DICOM routing rules can use tags in groups 0x8000 and above. This covers the site-wide routing rules and each DICOM receiver's routing expressions.
• XNAT-7945: The DICOM dump now includes elements that come after Pixel Data, and you can request any of them with ?field=. The summary view (summary=true) now shows the contents of sequences. The default dump keeps the same columns and order, with any new rows at the end. A ?field= value that isn't a valid tag or keyword now returns 400 (1.10.1 returned 200 with an empty table).
• XNAT-8803: DICOM mappings can use tags that come after Pixel Data, including private tags in groups 0x8000 and above.

Generate Snapshots for More Kinds of Scans

• XNAT-6581: Snapshots and thumbnails render for JPEG-compressed scans: JPEG Baseline and Extended, JPEG Lossless, JPEG-LS and JPEG 2000. This requires the OpenCV native library (see Before You Upgrade); uncompressed and RLE scans don't need it. If the library is missing, logging shows Cannot load OpenCV native library, followed by an UnsatisfiedLinkError that names org.opencv.core.Mat.
• XNAT-6743: Snapshots render for series that mix single-frame and multi-frame instances, such as whole-slide imaging.
• XNAT-6607, XNAT-8800: Snapshots render for scans whose objects all went to the secondary DICOM catalog: SOP classes XNAT doesn't treat as primary imaging, such as Secondary Capture, Segmentation, Parametric Map and Legacy Converted Enhanced images, when the separateSecondaryDicomOnArchive site preference is on (the default). XNAT counts frames from the DICOM headers when the catalog doesn't record them, without changing the catalog. As for other scans, viewing one for the first time adds a SNAPSHOTS resource to it.
  • Montages stay in Instance Number order even when some catalog entries don't record one; XNAT reads it from the file. Newly archived secondary catalogs now record instance numbers. Montages that 1.10.1 already stored keep their order until their SNAPSHOTS resource is deleted.

DicomEdit 6.10.0

DicomEdit 6.10.0 is bundled with XNAT 1.10.2; no separate installation or upgrade step is required. Upgrade notes for script authors and Java developers are in dicom-edit6/UPGRADING.md.

• A new, more efficient pixel redaction engine for alterPixels replaces pixelmed. It streams uncompressed pixel data through a fixed-size buffer, a line at a time, so memory use doesn't grow with the image. Compressed objects are decoded and re-encoded one frame at a time, so they need memory for one decoded frame, regardless of number of frames.
• Transfer syntax: uncompressed objects, including Implicit VR and Big Endian, stay in their own syntax. JPEG Lossless, JPEG-LS Lossless, JPEG 2000 Lossless and JPEG XL Lossless objects are re-encoded in their own syntax and are unchanged outside the redacted area. Lossy objects, including JPEG-LS near-lossless and every object in the general JPEG 2000 syntax come back as Explicit VR Little Endian with Lossy Image Compression set to 01 and the method and ratio added, so the rest of the image isn't degraded further. RLE and HTJ2K Lossless objects come back as Explicit VR Little Endian with a logged warning, because dcm4che has no encoder for them; the same happens if re-encoding fails.
• Fill and pixel types: the requested fill value is used, where versions 6.9.1 and earlier always used 0, and floating-point pixel data is redacted.
• A rectangle entirely outside the image now leaves the object unchanged.
• For backward consistency: Burned In Annotation is set to NO, and the de-identification method "Burned in text blacked out" and code 113101 are added, only when pixels change and only if not already present.
• Objects that cannot be redacted exactly are rejected rather than redacted incorrectly. The import or archive fails, with the reason in the XNAT log:
  • Integer pixel data whose Bits Allocated is not a multiple of 8 (including 1-bit)
  • A 4:2:2 layout without 3 samples per pixel, or with samples wider than 8 bits
  • YBR_PARTIAL_420
  • Floating-point pixel data in a compressed syntax
  • A compressed object whose decoded pixel data would be too long for a Pixel Data element (>4 GiB)
  • A compressed syntax XNAT can't decode (MPEG-2, H.264, HEVC, JPEG 2000 Part 2), or any JPEG-family object when the OpenCV native library is missing
• Runtime: redaction writes scratch files to the JVM's temporary directory, or to the directory set by the JVM system property dicom.pixeledit.scratch.dir. Allow at least the size of the pixel data for uncompressed objects, and about twice the decoded size for compressed ones. Redacting JPEG, JPEG-LS, JPEG 2000, HTJ2K or JPEG XL data needs the OpenCV native library.
• Header anonymization is unchanged, with one exception: dcm4che 5.35.0's DICOM dictionary defines (0010,0011) as Person Names to Use Sequence, so a script that sets it to a text value no longer works.
• Scripts can declare version "6.10", which makes sure they run only on engines with the new redaction behavior; all existing 6.0–6.7 headers are still accepted, but will run using the installed engine version, as before.

Manage Direct Archive and the Prearchive

• XNAT-7944: Delete direct-archive sessions completely, including half-received studies such as a failed DQR import.
  • DELETE /xapi/direct-archive/{id} now removes the session's files as well as its tracking record, when the directory belongs to that session alone.
  • Sessions that are queued, building or archiving, or still receiving files, are protected: deleting one returns 409. Triggering one manually with POST /xapi/direct-archive/{project}/{tag}/{name} returns 409 unless the session is receiving or in error.
  • Site admins can add force=true to either request for a session left queued, building or archiving by a failure or restart. The delete then removes it, and the trigger re-queues it.
• XNAT-8806: Non-admin users with All Data Access can view every prearchive session, read-only. They can open sessions, browse scans and resources, download files and view DICOM headers. On the session details page, the archive, change-project and delete actions aren't shown to them. (#69)

Other Updates and Bugfixes

• GitHub #70: Prearchive imports that merge into an existing session are no longer blocked by a file-locking error when catalogs are accessed concurrently.
• GitHub #78: XNAT no longer deadlocks when several requests use a data type that hasn't been used yet: right after a restart, on a rarely used type's first use, or after an admin creates a data type. The deadlock stopped all requests on the node until it was restarted. Concurrent requests also no longer drop fields from a data type's cached field list.
• GitHub #52: File history and the change summary track same-named files in different resources or folders separately, show each file's path, and keep the history of resources attached to a session.
• GitHub #64: The page shown when someone first signs in through an external provider (such as LDAP or OpenID) without a linked XNAT account has cleaner spacing, and its registration form states the actual username rules.
• The XAPI Swagger UI is updated to Springfox 2.10.5 and works when XNAT is deployed under a context path other than ROOT.
• With commons-fileupload 1.6.0, a multipart upload fails if the headers of one part exceed 512 bytes, which in practice means a file name of roughly 400 characters or more.

Plugin Developer Notes

Removed or changed APIs

A plugin built against 1.10.1 that uses any of these must be updated or rebuilt.

• DicomEdit 6.10.0: alterPixels now uses StreamingRectanglePixelEditHandler. The org.nrg.dicom.dicomedit.pixels.impl classes PixelmedPixelEditHandler, DicomImageBlackout (with its BurnedInAnnotationFlagAction enum) and Compressor are removed. PixelEditHandler is unchanged. (#43, #76)
• DirectArchiveSessionService and DirectArchiveSessionHibernateService add abstract interface methods, so a plugin that implements either interface must add them. The 1.10.1 signatures are unchanged; the two delete(long, UserI) methods are deprecated and still remove only the record. (#62, #81)
• Springfox (Swagger) is updated from 2.9.2 to 2.10.5, so a plugin that defines its own Docket must be updated. This also brings spring-plugin-core 2.0.0 (from 1.2.0), mapstruct 1.3.1 (from 1.2.0) and classgraph 4.1.7.
• org.nrg.Unpacker.unpack(File) and unpack(File, File), and Unzipper, Untarrer and Uncompressor.unpack(File, File), now return boolean instead of void. A plugin compiled against 1.10.1 that calls them must be rebuilt, and one that subclasses Unpacker must be updated.
• ChangeSummaryBuilderA.contains(...) takes an additional parent argument (#52).

Libraries

• dcm4che 5 is updated from 5.33.1 to 5.35.0, and the WAR now includes dcm4che-imageio-opencv and dcm4che-imageio-rle 5.35.0 and weasis-core-img 5.0.0. (#43)
  • dcm4che 2's RLE codec (dcm4che:dcm4che-imageio-rle 2.0.29) is no longer in the WAR; the dcm4che 5 RLE codec replaces it. A plugin that decodes RLE through dcm4che 2 should move to dcm4che 5. The dcm4che 2 artifacts, pixelmed and ImageJ are planned for removal in 1.11.
  • pixelmed is no longer a transitive dependency of DicomEdit. XNAT still ships the pixelmed jars at runtime so existing plugins load, but a plugin that compiles against pixelmed should declare it; the XNAT parent BOM still manages its version.
  • Applications that use DicomEdit outside XNAT must add org.dcm4che:dcm4che-imageio-opencv and dcm4che-imageio-rle 5.35.0, plus the OpenCV native library, to redact compressed data.
• Other library updates: Apache Ant 1.10.18 (from 1.9.8), commons-io 2.19.0 (from 2.15.1), commons-fileupload 1.6.0, Spring Security 5.7.14 and Spring LDAP 2.4.4.

New APIs and deprecations

• DicomAttributeIndex (org.nrg.dcm, dicomtools) gains a default getMaxTag(). Existing implementations compile unchanged. An implementation that can reference a tag after Pixel Data must override it to return that tag, or the session builder won't read far enough to find it. (#51)
• XNAT-8719: Use org.nrg.dicom.mizer.objects.Dcm4cheConvert.extractFmiFromDataset(), which documents that it removes the file meta information from its argument. splitFmiAndDataset() still works, but is deprecated and will be removed in a future release. (#63)