Summary

This release optimises performance in Databricks for incremental models in the derived schemas.

Features

• Optimize performance in Databricks for incremental models (Close #113)

Upgrading

To upgrade simply bump the snowplow-web version in your packages.yml file. If you want to take advantage of the performance optimisations in already existing incremental tables in your Databricks environment, run the following commands against Databricks for all of the incremental tables in your derived schema:

ALTER TABLE {derived_schema}.{table_name} SET TBLPROPERTIES (delta.autoOptimize.optimizeWrite = true, delta.autoOptimize.autoCompact = true);

By default, these are the snowplow_web_page_views, snowplow_web_sessions, and snowplow_web_users tables in the derived schema.