Overview:
In this release we introduce support for AWS S3 file storage, providing Dataverse installations with a cloud option. We also include support for Large Data upload via rsync and integration with an external application, the Data Capture Module (DCM). Other enhancements include improved Swift object storage, csv file ingest improvements, support for increased password complexity, downloading large guestbooks, removal of a user's roles, improved documentation, and various bug fixes.
- Provide S3 storage driver
- Improved Swift storage driver
- Support for large data upload and download workflows using rsync and DCM
- Improved CSV ingest.
- Configurable password complexity
- Fix and improve downloading of data for large guestbooks
- Fixed problem publishing a dataset with unescaped characters in title
- Disable user by button to remove all assigned roles
- Submit for review API endpoint
- Return to author API endpoint
- Create public-only configuration setting
- Restrict file API endpoint
- Improve migration documentation
- Improve User Guide section on permissions
- Provide installation-wide setting to control which metadata fields appear at top of dataset page
As always, thanks to all members of the Dataverse Community who contributed to this release by submitting suggestions, code, or other changes. Special thanks to Brian Silverstein, Oscar Smith, Rohit Bhattacharjee, and Sarah Ferry for their work on CSV and S3/Swift. Thanks to Pete Meyer for all the work on Rsync. Thanks to Don Sizemore and Akio Sone from Odum for fixes having to do with Glassfish and SPSS. Thanks to Solomon HM for improving migration documentation. Thanks to Jacob Makar-Limanov for work on UI accessibility.
For the complete list of issues, see the 4.8 milestone in Github.
For help with upgrading, installing, or general questions please email support@dataverse.org.
Installation:
If this is a new installation, please see our Installation Guide.
Upgrade:
If you are upgrading from v4.x, you must upgrade to each intermediate version before installing this version. When upgrading from the previous version, you will need to do the following:
- Undeploy the previous version.
- /glassfish4/bin/asadmin list-applications
- /glassfish4/bin/asadmin undeploy dataverse
- Stop glassfish and remove the generated directory, start
- service glassfish stop
- remove the generated directory: rm -rf /usr/local/glassfish4/glassfish/domains/domain1/generated
- service glassfish start
- Deploy this version.
- /glassfish4/bin/asadmin deploy dataverse-4.8.war
- Run the database update script.
psql -U <db user> -d <db name> -f upgrade_v4.7.1_to_v4.8.sql - Run the workflow script.
psql -U <db user> -d <db name> -f 3561-update.sql - Update citation.tsv to add grant agency and number to facet and advanced search.
curl http://localhost:8080/api/admin/datasetfield/load -X POST --data-binary @citation.tsv -H "Content-type: text/tab-separated-values"
Change in behavior note:
In this release, when a dataset is submitted for review by an author, the dataset no longer may be edited by the curator while it is in review. It must be returned to author before it can be modified.
If you are upgrading from v3.x, you will need to perform a migration to v4.x since our application was redesigned and the database schema are completely different. This is a significant undertaking. Please contact us (support at dataverse.org) before beginning. Also refer to our migration google group for additional support and information: https://groups.google.com/forum/#!forum/dataverse-migration-wg
IMPORTANT: If you are running TwoRavens with your dataverse:
Make sure the two applications are using the same version of the "pre-processed statistics" R code. Compare the 2 files:
On the TwoRavens side:
.../dataexplore/rook/preprocess/preprocess.R
On the Dataverse side:
.../applications/dataverse-4.8/WEB-INF/classes/edu/harvard/iq/dataverse/rserve/scripts/preprocess.R
If they are different, replace the Dataverse copy with the TwoRavens copy (i.e., the TwoRavens version wins!).
And, also, remove all the already-generated pre-processed fragments in your Dataverse file directory, for example:
cd [files directory]
rm -f `find . -name '*.prep'`
If the two copies are the same, you don't need to do any of this.
Please note that this is a temporary measure, we are working on a fix that will make the two applications resolve code version conflicts like this automatically.