Dataverse 6.9

Please note: To read these instructions in full, please go to https://github.com/IQSS/dataverse/releases/tag/v6.9 rather than the list of releases, which will cut them off.

This release brings new features, enhancements, and bug fixes to Dataverse. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project!

Release Highlights

Highlights for Dataverse 6.9 include:

• Quickstart Guide
• Role Assignment History Tracking
• Scaling Dataverse with Data Size (Admin Guide)
• Storage Quotas on Individual Datasets
• Additional Licenses
• DataCite Scaling
• Support for COAR Notify Relationship Announcement
• Infrastructure upgrade: Payara
• New and improved APIs
• Bug fixes

Features Added

Quickstart Guide

A new Quickstart Guide has been added to help researchers understand what Dataverse is, how to publish datasets and collections, and how to find data. Buttons have been added to the top of the User Guide to direct people to these pages. Feedback on this new guide is welcome, such as through Google Groups, Zulip, or the Documentation Working Group. Special thanks to Dieuwertje Bloemen for writing most of the guide and to Vaida Plankytė for contributing. See https://guides.dataverse.org/en/6.9/quickstart and #11653.

Role Assignment History Tracking

Dataverse can now track the history of role assignments, allowing administrators to see who assigned or revoked roles, when these actions occurred, and which roles were involved. This feature helps with auditing and understanding permission changes over time. An additional column called "Role Assignment History" has been added to the permission management page for collections. The information can also be downloaded via API in CSV and JSON formats. This feature is off by default but can be enabled with the "role-assignment-history" feature flag. See the User Guide, API Guide, and #11612.

Scaling Dataverse with Data Size (Admin Guide)

A new section, Scaling Dataverse with Data Size, has been added to the Admin Guide. It is intended to help administrators configure Dataverse appropriately to handle larger amounts of data. See https://guides.dataverse.org/en/6.9/admin/big-data-administration.html and #11850.

Storage Quotas on Individual Datasets

It is now possible to define storage quotas on individual datasets via API. The practical use case is for datasets in the top-level root collection. This feature does not address the use case of a user creating multiple datasets (#11529). A convenience API /api/datasets/{id}/uploadlimits has been added to show the "remaining storage" and "number of files" quotas, if present. See the guides, #11987, and #11997.

Additional Licenses

The following Open Data Commons licenses have been added:

• Open Database License (ODbL)
• Open Data Commons Attribution License (ODC-By)
• Open Data Commons Public Domain Dedication and License (PDDL)

The following software license has been added:

• European Union Public License (EUPL)

The following country-specific license has been added:

• Open Government Licence (OGL UK)

The licenses above are widely recognized and used in Europe and beyond to promote data and software sharing. See the guides and #11522.

DataCite Scaling

Dataverse now retries calls to DataCite when their server is overloaded or when the Dataverse server has hit the DataCite rate limit.

It also introduces an option to only update DataCite metadata after checking to see if the current DataCite information is out of date. (This adds a request to get information from DataCite before any potential write of new information which will be more efficient when most DOIs have not changed but will result in an extra call to get info when a DOI has changed.) This functionality is off by default but can be enabled with the only-update-datacite-when-needed feature flag.

Both of these can help when DataCite is being used heavily, e.g. creating and publishing datasets with many datafiles and using file DOIs, or doing bulk operations that involve DataCite with many datasets.

See the guides and #11832

Support for COAR Notify Relationship Announcement

Dataverse now supports sending and receiving Linked Data Notification messages involved in the COAR Notify Relationship Announcement Workflow.

Dataverse can send messages to configured repositories announcing that a dataset has a related publication (as defined in the dataset metadata). This may be done automatically upon publication or triggered manually by a superuser. The receiving repository may do anything with the message, with the default expectation being that the repository will create a backlink from the publication to the dataset (assuming the publication exists in the repository, admins agree the link makes sense, etc.)

Conversely, Dataverse can receive notices from other configured repositories announcing relationships between their publications and datasets. If the referenced dataset exists in the Dataverse instance, a notification will be sent to users who can publish the dataset, or, optionally, only superusers who can publish the dataset. They can then decide whether to create a backlink to the publication in the dataset metadata.

See the guides, #8914, and #10490. (Earlier releases of Dataverse had experimental support in this area that was based on message formats defined prior to finalization of the COAR Notify specification for relationship announcements.)

Other Features Added

• When creating a dataset, the "Host Dataverse" field is not shown when the user can add datasets to only one collection. See #11865.
• In the UI for granting file access, restricted files in draft will now show "Draft/Unpublished". See #7618 and #11794.
• This version of Dataverse includes extensions of the Dataverse external vocabulary mechanism that improve Dataverse's ability to include metadata about vocabulary terms and external identifiers such as ORCID and ROR in its metadata exports. More information on how to configure external vocabulary scripts to use this functionality can be found in docs/readme.md and in the examples in the https://github.com/gdcc/dataverse-external-vocab-support repository. See #11793.
• The external vocabulary mechanism now supports assigning metadatablock dataset field types of fieldType textbox (multiline inputs) as managed fields. See #11954.
• It is now possible to configure all database settings at once with a JSON file. See the guides, #11639, and #11654.
• CORS support has been modernized so browser integrations (previewers, external tools, JS clients) work correctly with multiple origins and proper caching. See the guides, #11744, and #11745.
• Integration with Local Contexts has been updated to support the change in their API regarding how DOIs entered as "Optional Project Information" are represented. See the guides and #11904.
• This release adds database indexes on GuestbookResponse and DatasetMetrics, speeding up Dataset deletes. It also adds a constraint preventing null VersionState, as a matter of good housekeeping practice. See #11828 and #11898.
• Permission reindexing, which occurs, e.g., after a user has been granted a role on a collection, has been made faster and less memory intensive in this release. See #11822.
• Performance has been improved when retrieving citations from DataCite. A related setting called dataverse.api.mdc.min-delay-ms has been added. See the guides, #11777, and #11781.
• Processing of comma-separated lists in settings has been centralized and now ignores spaces around commas. See the guides and #11745.

Bugs Fixed

• Editing a controlled vocabulary field (i.e. one with values specified in the field's metadatablock) that only allows a single selection would also update the value in the prior published version if (and only if) the edit was made starting from the published version (versus an existing draft). This is now fixed. The bug appears to be 11+ years old and previously unreported. As the value in the database was overwritten, there is no simple way to detect if or when this occurred without looking at backups or archival file copies. See #11990 and #11991.
• In prior versions of Dataverse, publishing a dataset via the superuser-only update-current-version option would not set the current curation status (if enabled/used) to none/empty and, in v6.7, would not maintain the curation status history. These issues are now resolved and the update-current-version option works the same as normal publication of a new version with regard to curation status. See #11783 and #11784.
• This release fixes problems with guestbook questions being displayed at download when files are selected from the dataset files table when guestbook-at-request is enabled and not displaying when they should when access is requested from the file page. See #11800, #11808, and #11835.
• The optional Croissant exporter has been updated to 0.1.6 to prevent variable names, variable descriptions, and variable types from being exposed for restricted files. See gdcc/exporter-croissant#20 and #11752.
• Manage Guestbooks page was optimized to load much faster for collections with large numbers of downloads recorded.

API Updates

• The Settings API no longer accepts keys (in the sense of key/value pairs) that are not defined in the code. See "Database Settings Cleanup" in these release notes for details, #11639 and #11654.
• A new API endpoint has been added for retrieving a list of collections to which a given dataset or collection can be linked. See the guides, #11710, and #11741.
• A new API endpoint has been added to manage dataset licenses. See the guides, #11771, #11815, and #11958.
• A new API endpoint has been added to manage dataset terms of access for restricted files. See the guides, #11772, and #11893.
• A new API endpoint has been added for getting and setting the metadata language of a collection. See the guides, 11856, and #11857.
• The /api/dataverses/{identifier}/templates endpoint previously required editDataverse permissions to retrieve the list of dataverse templates. This has been corrected. The endpoint now addDataset permissions instead. See #11796 and #11801.
• The endpoint /api/datasets/{id}/versions/compareSummary was previously returning an incorrect count for
  the changedFileMetaData field. The logic for calculating this count has been fixed to accurately reflect the total number of file metadata changes across all files in the dataset version. See #11921 and #11944.
• The "File Version Differences" and "Dataset Version Summaries" API endpoints have been improved with pagination support (with limit and offset parameters), performance improvements, and a bug fix. See the guides, #11561, #11855 and #11859.
  The Notifications API has been improved (with onlyUnread, limit, and offset parameters) but also has some breaking changes. See the guides, #11804, #11851, and #11854.
• The storage API driver endpoint now returns a JSON object with the driver's "name", "type" and "label", and booleans indicating whether the driver has "directUpload", "directDownload", and/or "uploadOutOfBand" enabled. This change also affects the /api/admin/dataverse/{dataverse-alias}/storageDriver api call. In addition, this call now supports an optional ?getEffective=true to find the effective storageDriver (the driver that will be used for new datasets in the collection), See the guides, #11695, and #11716.

Security Updates

This release contains important security updates such as an upgrade to Payara (#11827) and how CORS is handled (#11745). If you are not receiving security notices, please sign up by following the steps in the guides.

Backward Incompatible Changes

Generally speaking, see the API Changelog for a list of backward-incompatible API changes.

• This release of Dataverse requires an upgrade to Payara. Please be aware that you need to upgrade Payara and Dataverse at the same time because older versions of Dataverse are not compatible with newer versions of Payara. See upgrade instructions for details.
• CORS is no longer enabled by default. See the upgrade instructions for details as well as #11745.
• See the "Database Settings Cleanup" section below. In the past, the settings API would accept any key and value. This is no longer the case because validation has been added. Also the way to set per-format size limits for tabular ingest has changed. JSON input is now used. See #11639 and #11654.
• The Update Collection Input Levels API endpoint no longer deletes the custom input levels previously modified for the given collection. In order to update a previously modified custom input level, it must be included in the JSON provided to the api. See the guides, #11387, and #11748.
• For /api/admin/dataverse/{dataverse-alias}/storageDriver and /api/datasets/{identifier}/storageDriver the driver name is no longer returned in data.message. Instead, it is returned as data.name (along with other information about the storageDriver). See the guides, #11695, and #11664.
• The POST /api/admin/makeDataCount/{id}/updateCitationsForDataset processing is now asynchronous and the response no longer includes the number of citations. The response can be OK if the request is queued or 503 if the queue is full (default queue size is 1000). See #11777 and #11781.

End-Of-Life (EOL) Announcements

PostgreSQL 13 Reached EOL on 13 November 2025

We mentioned this in the Dataverse 6.6 and 6.8 release notes, but as a reminder, according to https://www.postgresql.org/support/versioning/ PostgreSQL 13 reached EOL on 13 November 2025. As mentioned in the Installation Guide, we recommend running PostgreSQL 16 since it is the version we test with in our continuous integration (since February 2025). The Dataverse 5.4 release notes explained the upgrade process from 9 to 13 (e.g. pg_dumpall, etc.) and the steps will be similar. If you have any problems, please feel free to reach out (see "getting help" in these release notes).

Payara 6 Reaching EOL with 6.2025.11

As explained in #12020 and https://payara.fish/blog/payara-6-community-end-of-life/ Payara 6.2025.11 is the last release in the 6.x series. A pull request at #12043 to upgrade to Payara 7 has already been made but won't make it into this release of Dataverse. You are welcome to try it out and give feedback.

Developer Updates

• The ExportDataProvider framework in the dataverse-spi package has been extended, adding some extra options for developers of metadata exporter plugins. See the guides #11766, and #11767.

Database Settings Cleanup

With this release, we remove some legacy quirks around Database settings and provide better Admin API endpoints for managing settings.

Most important changes:

1. The setting BuiltinUsers.KEY was renamed to :BuiltinUsersKey to align with our naming pattern.
2. The setting WorkflowsAdmin#IP_WHITELIST_KEY was renamed to :WorkflowsAdminIpWhitelist to align with our naming pattern.
3. The setting :TabularIngestSizeLimit no longer uses suffixes for formats and becomes a JSON-based setting instead. See the guides for details.
4. If any of the settings above are set, they will be migrated to their new form automatically for you when the war file is deployed (Flyway migration).
5. You can no longer (accidentally) create or use arbitrary setting names or languages.
   All Admin API endpoints for settings now validate setting names and languages for existence and compliance.

As an administrator of a Dataverse instance, you can now make use of enhanced Bulk Operations on the Settings Admin API:

1. Retrieving all settings as JSON via GET /api/admin/settings supports localized options now, too.
2. You can replace all existing settings in an idempotent way sending JSON to PUT /api/admin/settings.
   This will create, update and remove settings as necessary in one atomic operation.
   The new endpoint is especially useful to admins using GitOps or other automations.
   It allows control over all Database Settings from a single source without risking an undefined state.

Note: Despite the validation of setting names and languages, the content of any database setting is still not being validated when using the Settings Admin API!

Important Considerations During Upgrade Of Your Installation

1. Running a customized fork? Make sure to add any custom settings to the SettingsServiceBean.Key enum before deploying!
2. Any database settings not contained in the SettingServiceBean.Key will be removed from your database during each deployment cycle.
3. As always when upgrading, make sure to backup your database beforehand!
   You can also use the existing API endpoint /api/admin/settings to retrieve all settings as JSONish data for a quick backup before upgrading.

See also #11639 and #11654.

New Settings

New JVM Options (MicroProfile Config Settings)

• dataverse.api.mdc.min-delay-ms
• dataverse.coar-notify.relationship-announcement.notify-superusers-only
• dataverse.feature.only-update-datacite-when-needed
• dataverse.feature.role-assignment-history
• dataverse.ldn.allowed-hosts

New Database Settings

• :COARNotifyRelationshipAnnouncementTargets
• :COARNotifyRelationshipAnnouncementTriggerFields

Updated Database Settings

The following database settings were added to the official list within the code (to remain valid with the settings cleanup mentioned above):

• :BagGeneratorThreads
• :BagItHandlerEnabled
• :BagItLocalPath
• :BagValidatorJobPoolSize
• :BagValidatorJobWaitInterval
• :BagValidatorMaxErrors
• :BuiltinUsersKey - formerly BuiltinUsers.KEY
• :CreateDataFilesMaxErrorsToDisplay
• :DRSArchiverConfig - a Harvard-specific setting
• :DuraCloudContext
• :DuraCloudHost
• :DuraCloudPort
• :FileCategories
• :GoogleCloudBucket
• :GoogleCloudProject
• :LDNAnnounceRequiredFields
• :LDNTarget
• :WorkflowsAdminIpWhitelist - formerly WorkflowsAdmin#IP_WHITELIST_KEY
• :PrePublishDatasetWorkflowId - formerly WorkflowServiceBean.WorkflowId:PrePublishDataset
• :PostPublishDatasetWorkflowId - formerly WorkflowServiceBean.WorkflowId:PostPublishDataset

Deleted Settings

• In Dataverse 6.7 (#11454) the :AllowCors setting was deprecated in favor of a new dataverse.cors.origin setting and now, in Dataverse 6.9, the :AllowCors setting has been removed (#11745). See also the step on CORS in the upgrade instructions below.
• BuiltinUsers.KEY - now :BuiltinUsersKey
• WorkflowsAdmin#IP_WHITELIST_KEY - now :WorkflowsAdminIpWhitelist
• WorkflowServiceBean.WorkflowId:PrePublishDataset - now :PrePublishDatasetWorkflowId
• WorkflowServiceBean.WorkflowId:PostPublishDataset - now :PostPublishDatasetWorkflowId

Complete List of Changes

For the complete list of code changes in this release, see the 6.9 milestone in GitHub.

Getting Help

For help with upgrading, installing, or general questions please see getting help in the Installation Guide.

Installation

If this is a new installation, please follow our Installation Guide. Please don't be shy about asking for help if you need it!

Once you are in production, we would be delighted to update our map of Dataverse installations around the world to include yours! Please create an issue or email us at support@dataverse.org to join the club!

You are also very welcome to join the Global Dataverse Community Consortium (GDCC).

Upgrade Instructions

NOTE: These instructions include an upgrade to Payara 6.2025.10. Do not attempt to run Dataverse 6.8 or lower on Payara 6.2025.10. It won't work. To see the changes that had to be made to the the Dataverse code to make it compatible with Payara 6.2025.10 see #11827.

Upgrading requires a maintenance window and downtime. Please plan accordingly, create backups of your database, etc.

When doing backups, make sure your settings are backed up. See "Database Settings Cleanup" above for details.

0. These instructions assume that you are upgrading from the immediate previous version. That is to say, you've already upgraded through all the 6.x releases and are now running Dataverse 6.8. See tags on GitHub for a list of versions. If you are running an earlier version, the only supported way to upgrade is to progress through the upgrades to all the releases in between before attempting the upgrade to this version.

If you are running Payara as a non-root user (and you should be!), remember not to execute the commands below as root. By default, Payara runs as the dataverse user. In the commands below, we use sudo to run the commands as a non-root user.

Also, we assume that Payara 6 is installed in /usr/local/payara6. If not, adjust as needed.

export PAYARA=/usr/local/payara6

(or setenv PAYARA /usr/local/payara6 if you are using a csh-like shell)

1. List deployed applications

$PAYARA/bin/asadmin list-applications

2. Undeploy the previous version (should match "list-applications" above)

$PAYARA/bin/asadmin undeploy dataverse-6.8

3. Stop Payara

sudo service payara stop

4. Upgrade to Payara 6.2025.10

The recommended Payara version has been updated to Payara 6.2025.10. See #11827.

As a reminder, Payara 6.2025.10 cannot be used with earlier versions of Dataverse, e.g. v6.8.

The steps below reuse your existing domain directory with the new distribution of Payara. You may also want to review the Payara upgrade instructions as it could be helpful during any troubleshooting:
Payara Release Notes.
We also recommend you ensure you followed all update instructions from the past releases regarding Payara.
(The most recent Payara update was for Dataverse 6.7.)

Move the current Payara directory out of the way:

mv $PAYARA $PAYARA.6.2025.3

Download the new Payara version 6.2025.10 (from https://www.payara.fish/downloads/payara-platform-community-edition/ or https://nexus.payara.fish/repository/payara-community/fish/payara/distributions/payara/6.2025.10/payara-6.2025.10.zip), and unzip it in its place:

cd /usr/local
unzip payara-6.2025.10.zip

Replace the brand new payara6/glassfish/domains/domain1 with your old, preserved domain1:

mv payara6/glassfish/domains/domain1 payara6/glassfish/domains/domain1_DIST
mv payara6.6.2025.3/glassfish/domains/domain1 payara6/glassfish/domains/

Once your old, preserved domain1 directory is in place, copy the *.p12 files from distributed domain1/config directory into it, at least if they are different. These files include base certs. Over time, using the old cert files can cause SSL errors. If the domain.xml file is old enough, it may also reference *.jks files instead of the *.p12 ones. If so, update domain.xml so that these lines reference *.p12 files instead of *.jks files.

cp payara6/glassfish/domains/domain1_DIST/*.p12 payara6/glassfish/domains/domain1

5. Download and deploy this version

wget https://github.com/IQSS/dataverse/releases/download/v6.9/dataverse-6.9.war
$PAYARA/bin/asadmin deploy dataverse-6.9.war

Please note that, depending on your database, the initial deployment of this version may take a little longer than usual. This is because new database indexes will need to be created in real time. The time it should take is not even a function of the overall size of your database, but specifically the GuestbookResponse table. (In other words, not a function of how much content you have in the repository, but how popular it is).

6. For installations with internationalization or text customizations:

Please remember to update translations via Dataverse language packs.

If you have text customizations you can get the latest English files from https://github.com/IQSS/dataverse/tree/v6.9/src/main/java/propertyFiles.

7. Enable or re-enable CORS, if desired

As of Dataverse 6.8 CORS was enabled by default but this is no longer the case. See #11745.

If you are relying on CORS, e.g. using file previewers hosted at gdcc.github.io, and did not configure dataverse.cors.origin as recommended in the 6.7 release notes, you should do so now. With updates in 6.9, it is also possible to limit CORS support to specific internet hosts. See the guides for details.

8. Update Croissant exporter, if enabled, and reexport metadata

If you have enabled the Croissant dataset metadata exporter, you should upgrade to version 0.1.6.

• Stop Payara.
• Delete the old Croissant exporter jar file. It will be located in the directory defined by the dataverse.spi.exporters.directory setting.
• Download the updated Croissant jar from https://repo1.maven.org/maven2/io/gdcc/export/croissant/ and place it in the same directory.
• Restart Payara.
• Run reExportAll.