What's New
Complete Refactor of DKAN Datastore
The Feeds module has long been a key component of the DKAN Datastore. While it has served us well over the years as a community-supported framework for importing CSV and similar files into database tables, it has also added a lot of overhead and bloat to what should be a simple system. At the same time, much of the datastore code is written specifically to interact with Feeds, and separating the two proved to be impossible. The Datastore module has now been completely re-written to make it faster, more stable, and more modular. A properly object-oriented, decoupled architecture allows its various classes to be extended and plans to support additional options for datastore infrastructure (such as a second MySQL database or even a 3rd party service like Carto) are in the works.
New CSV importer w/o Feeds
A simple parser will try to import the entire file into the datastore, and continue in the background on the next cron run if the file is too big. Improvements can be found both in the code and in the UI. See the docs for more information.
Better API support
Datastore behaviors can also now be controlled through the Dataset REST API, so that you can automate actions like importing resources to the datastore and dropping datastore tables.
Note: While the system for getting data into the datastore has changed significantly, the API for querying the datastore remains the same.
Improvements to DKAN command-line tooling
While not a change to the core DKAN codebase, this release marks the release of a new command-line tool for working with DKAN, DKAN Tools. This will make it easier for anyone to stand up DKAN locally, manage Docker containers, and use different CI pipelines.
Support for non-date values in the dataset "modified" field
The "Modified Date" field has been converted to a text field to accommodate ISO 8601 repeating interval values such as R/P1D and R/P2W. Previously, this field had been a date field, which was incompatible with certain values that would be allowed in Project Open Data's modified field.
Support PHP 7
Many great performance improvements happened in PHP 7. DKAN has been updated to be compatible with php 7.1 and allows users to take advantage of those improvements. Our CI and testing infrastructure has also been updated to use PHP 7.1.
Changelog
- #2179 Refactor Datastore Importer
- #2627 Add support for ISO-8106 duration values in "modified" when harvesting
- #2729 Update recline to 2.1
- #2713 CSV Parser Improvements: trailing commas & escape chars
- #2740 Fix dkan_workflow anonymous access to revisions and unpublished content
- #2724 Update contrib modules:
- better_exposed_filters
- file_entity
- media
- panopoly_widgets
- search_api_db
- uuid
- date: fixes issue 2843367
- #2718 Update odsm_dkan token values for remote file
- #2707 DKAN workflow documentation updates
- #2723 Include dkan.profile to prevent undefined function in update
Deployment Steps
drush rrdrush updb -ydrush cc all -y