Dataverse 5.3

This release brings new features, enhancements, and bug fixes to Dataverse. Thank you to all of the community members who contributed code, suggestions, bug reports, and other assistance across the project.

Release Highlights

Auxiliary Files (Experimental)

Auxiliary files can now be added to datafiles and accessed using new experimental API endpoints. These endpoints allow additional, non-Dataverse generated metadata to be added alongside datafiles in dataverse.

The support for auxiliary files in Dataverse is being driven by integration with the Open Differential Privacy (DP) Project and is designed to support the deposit and retrieval of differentially private metadata, but the endpoints are not specific to differential privacy use cases.

Additional Banner Functionality

Banners in Dataverse can now be set to allow dismissal by a logged in user. Previously, banners would persist until they were removed by an administrator. This allows administrators to more easily communicate one-time messages to users.

File Tags Searchable from Advanced Search and Dataset Search

File tags ("Documentation", "Data", "Code", etc.) now appear on the Advanced Search page.

Performing a search for files on the dataset page now includes file tags. Previously, only file name and file description were searched.

Easier Configuration of Database Connections

Previously, the configuration of the database connections has been quite static and not very easy to update. This has been an issue especially for cloud and container usage. Using new technologies provided by the move to Payara, you can now more easily configure the connection to your PostgreSQL DB.

Using MicroProfile Config API (Issue #7000, Issue #7418), you can much more easily specify configuration
details. For an overview of supported options, please see the
Installation Guide.

Note that some settings have been moved from domain.xml to code, such as min and max pool size.

Major Use Cases

Newly-supported use cases in this release include:

• Users can use an API to add auxiliary files to files in order to provide metadata representations for specific tools or integrations (Issue #7275, PR #7350)
• Administrators can use a new API to manage banner messages and take advantage of new banner display options (Issue #7263, PR #7434)
• Users replacing files will now have their files renamed when a file name conflict exists, making the behavior consistent with upload and edit (Issue #7335, PR #7336)
• Users will now be able to search on file tags on the advanced search and dataset pages (Issue #7194, PR #7385)

Notes for Dataverse Installation Administrators

Payara 5.2020.6 (or Higher) Required

Some changes in this release require an upgrade to Payara 5.2020.6 or higher.

Instructions on how to update can be found in the
Payara documentation

New Banner API, Obsolete DB Settings

The functionality previously provided by the DB settings :StatusMessageHeader and ::StatusMessageText is no longer supported and is now provided through the Manage Banner Messages API. Learn more in the API Guide.

New Database Settings and JVM Options

Several new JVM options have been added in this release:

• dataverse.db.name
• dataverse.db.user
• dataverse.db.password
• dataverse.db.host
• dataverse.db.port

For an overview of these new options, please see the
Installation Guide

See above note about obsolete DB options.

Introducing MicroProfile Config API

With this Dataverse release, Dataverse Administrators can start to make use of the MicroProfile Config API.

This will benefit both developers and sysadmins, but the codebase will have to be refactored to make use of it. As this will take time, we will always provide a backward compatible way of using it.

For more details about these new options, please see the Consuming Configuration section of the Developer Guide.

Java Message System Configuration

The Ingest process uses the Java Message System to create ingest tasks in a queue. That queue had been configured from command line or domain.xml before. This has now changed to being done
in code.

In the unlikely case you might want to change any of these settings, feel free to change and recompile or raise an issue on Github. See IngestQueueProducer for more details.

If you want to clean up your existing installation, you can delete the old, unused queue like this:

• <payara install path>/bin/asadmin delete-connector-connection-pool --cascade=true jms IngestQueueConnectionFactoryPool

Notes for Tool Developers and Integrators

Experimental Auxiliary File Support

Experimental endpoints have been added to allow auxiliary files to be added to datafiles. These auxiliary files can be deposited and accessed via API. Later releases will include options for accessing these files through the UI. For more information, see the Auxiliary File Support section of the Developer Guide.

Complete List of Changes

For the complete list of code changes in this release, see the 5.3 Milestone in Github.

For help with upgrading, installing, or general questions please post to the Dataverse Google Group or email support@dataverse.org.

Installation

If this is a new installation, please see our Installation Guide.

Upgrade Instructions

0. These instructions assume that you've already successfully upgraded from Dataverse 4.x to Dataverse 5 following the instructions in the Dataverse 5 Release Notes.

1. Upgrade to Payara 5.2020.6 or higher.

Instructions on how to update can be found in the
Payara documentation.

It would likely be safer to upgrade Payara first, while still running Dataverse 5.2, and then proceed with the steps below. Upgrading from an earlier version of Payara should be a straightforward process: Undeploy Dataverse; stop Payara; move the current Payara directory out of the way; unzip the new Payara version in its place; replace the brand new payara/glassfish/domains/domain1 with your old, preserved domain1; start Payara, deploy Dataverse 5.2. We still recommend that you read the detailed upgrade instructions above; and, if you run into any issues with this upgrade, it will help to be able to separate them from any problems with the upgrade of Dataverse proper.
If you are still using pre-5.0 version of Dataverse, and Glassfish version 4, please follow the upgrade instructions in the Dataverse 5.0 release notes; but use the latest version of Payara 5 (5.2020.7, as of this writing).

2. Undeploy the previous version.

• <payara install path>/bin/asadmin list-applications
• <payara install path>/bin/asadmin undeploy dataverse<-version>

(where <payara install path> is where Payara 5 is installed, for example: /usr/local/payara5)

3. Update your database connection.

Please configure your connection details, replacing all the ${DB_...}.

• <payara install path>/bin/asadmin create-system-properties "dataverse.db.user=${DB_USER}"

• <payara install path>/bin/asadmin create-system-properties "dataverse.db.host=${DB_HOST}"

• <payara install path>/bin/asadmin create-system-properties "dataverse.db.port=${DB_PORT}"

• <payara install path>/bin/asadmin create-system-properties "dataverse.db.name=${DB_NAME}"

• echo "AS_ADMIN_ALIASPASSWORD=${DB_PASS}" > /tmp/password.txt

• <payara install path>/bin/asadmin create-password-alias --passwordfile /tmp/password.txt dataverse.db.password

• rm /tmp/password.txt

4. In domain.xml, verify that the __TimerPool jdbc-connection-pool is using the H2 database, as follows (if you have the old Derby version from Glassfish 4, replace it):

<jdbc-connection-pool datasource-classname="org.h2.jdbcx.JdbcDataSource" name="__TimerPool" res-type="javax.sql.XADataSource"> <property name="URL" value="jdbc:h2:${com.sun.aas.instanceRoot}/lib/databases/ejbtimer;AUTO_SERVER=TRUE"></property> </jdbc-connection-pool>

5. Reset the EJB timer database back to default:

• <payara install path>/bin/asadmin set configs.config.server-config.ejb-container.ejb-timer-service.timer-datasource=jdbc/__TimerPool

6. Delete the old password alias and DB pool:

• <payara install path>/bin/asadmin delete-jdbc-connection-pool --cascade=true dvnDbPool
• <payara install path>/bin/asadmin delete-password-alias db_password_alias

7. Stop payara, remove the generated and ejbtimer database directories, then restart.

• service payara stop
• rm -rf <payara install path>/glassfish/domains/domain1/generated
• rm -rf <payara install path>/glassfish/domains/domain1/lib/databases/ejbtimer
• service payara start

8. Deploy this version.

• <payara install path>/bin/asadmin deploy dataverse-5.3.war

9. Restart payara

• service payara stop
• service payara start