OHDSI/WebAPI v2.8.0

on GitHub

New Features

Important note: The 2.8 line of WebAPI will be the last release to support Oracle and MS Sql as the WebAPI database platform. (Note CDM databases are not impacted by this). So, for future major/minor releases, you will need to install a Postgres instance. We can look into community driven support for other platforms, but going forward official support will be limited to Postgresql for WebAPI.

Added validation mechanism to different domain entities of WebAPI: IR Analysis, Cohort Characterization, Pathways, etc). These validation messages are presented in the Atlas UI.

Cohort Characterization: new design element 'Distribution Criteria' allowing the creation of distribution statistics from criteria expressions. (#1449)

Data Sources - Data Density: Added Measurement data over time to Data Density Plot. (#1620)

Cohort Pathways: Allow Repeats option allows pathway segments to repeat in the result. By default, this is disabled. (#1692)

Geospatial functionality (#1445)

Use Dockerfile to facilitate development (#1384)

Cohort Sampling: samples can be created for a given cohort definition on a specific source. (#1657) (Thank you Andries Purnal (Andries.Purnal@ucb.com), Florence Giesen (Florence.Giesen@ucb.com), @MaximMoinat and The Hyve

Vocabulary Service: Added endpoitns for cocnept ancestor (#1664)

Server-Side rendering of cohort expressions: Circe expressions are converted into Markdown using Freemarker templates (#1655).

R packages for PLE/PLP were updated to use Hydra (#1643)

Implemented server-side exports of concept sets for Feature Analysis and Cohort Characterization (#1461)

Individual auth providers can be disabled. Additional information can be found in README.md. (#1623)

Several performance improvements on exporting and results retrieval (#1005, #1010)

Notable Fixes

Due to the long duration between 2.7 and 2.8 release, there were too many fixes to list here. A list of fixes can be found here (https://github.com/OHDSI/WebAPI/issues?q=is%3Aissue+is%3Aclosed+milestone%3AV2.8.0)

Migration Notes

Use SqlRender to translate the following SQL for your specific environment and run on all CDM results schemas:

Cohort Caching

Per #1353, users will need to update the results schema to include the new *_cache tables, specifically:

ALTER TABLE @results_schema.cohort_inclusion ADD design_hash int NULL;

ALTER TABLE @results_schema.cc_results ADD aggregate_id int NULL;
ALTER TABLE @results_schema.cc_results ADD aggregate_name varchar(1000);
ALTER TABLE @results_schema.cc_results ADD missing_means_zero int NULL;

IF OBJECT_ID('@results_schema.cohort_cache', 'U') IS NULL
CREATE TABLE @results_schema.cohort_cache (
	design_hash int NOT NULL,
	SUBJECT_ID bigint NOT NULL,
	cohort_start_date date NOT NULL,
	cohort_end_date date NOT NULL
);

IF OBJECT_ID('@results_schema.cohort_censor_stats_cache', 'U') IS NULL
CREATE TABLE @results_schema.cohort_censor_stats_cache (
  design_hash int NOT NULL,
  lost_count BIGINT NOT NULL
);

IF OBJECT_ID('@results_schema.cohort_inclusion_result_cache', 'U') IS NULL
CREATE TABLE @results_schema.cohort_inclusion_result_cache (
  design_hash int NOT NULL,
  mode_id int NOT NULL,
  inclusion_rule_mask bigint NOT NULL,
  person_count bigint NOT NULL
);

IF OBJECT_ID('@results_schema.cohort_inclusion_stats_cache', 'U') IS NULL
CREATE TABLE @results_schema.cohort_inclusion_stats_cache (
  design_hash int NOT NULL,
  rule_sequence int NOT NULL,
  mode_id int NOT NULL,
  person_count bigint NOT NULL,
  gain_count bigint NOT NULL,
  person_total bigint NOT NULL
);

IF OBJECT_ID('@results_schema.cohort_summary_stats_cache', 'U') IS NULL
CREATE TABLE @results_schema.cohort_summary_stats_cache (
  design_hash int NOT NULL,
  mode_id int NOT NULL,
  base_count bigint NOT NULL,
  final_count bigint NOT NULL
);

IF OBJECT_ID('@results_schema.cohort_sample_element', 'U') IS NULL
CREATE TABLE @results_schema.[cohort_sample_element] (
    [cohort_sample_id] int NOT NULL, 
    [rank_value] int NOT NULL, 
    [person_id] bigint NOT NULL, 
    [age] int NULL, 
    [gender_concept_id] int NULL
);

New cache settings in pom.xml:

cache.generation.invalidAfterDays=${cache.generation.invalidAfterDays} 
cache.generation.cleanupInterval=${cache.generation.cleanupInterval}
cache.generation.useAsync=${cache.generation.useAsync} 
(cache.generation.useAsync details: https://github.com/OHDSI/WebAPI/pull/1459)

Cohort Sampling

IF OBJECT_ID('@results_schema.cohort_sample_element', 'U') IS NULL
CREATE TABLE @results_schema.cohort_sample_element(
    cohort_sample_id int NOT NULL,
    rank_value int NOT NULL,
    person_id bigint NOT NULL,
    age int,
    gender_concept_id int
);

Cohort Characterization

The following alters the existing cc_results table, the full definition can be found here (https://github.com/OHDSI/WebAPI/blob/master/src/main/resources/ddl/results/cohort_characterizations.sql)

alter table@results_schema.cc_results ADD aggregate_id int NULL, aggregate_name varchar(1000), missing_means_zero int NULL;

Acknowledgements

Thanks to @pavgra, @ssuvorov-fls, @aklochkova, @AntonStepanof, @chrisknoll, @wivern, @anton-abushkevich, @ekaterinakrivets, @YaroslavTir, @SergeySukhanov, @KSadomtsev, @konstjar, @vlbe, @AnthonyMolinaro, @MaximMoinat (and the team at TheHyve) for their contributions and reviews as part of this release.