This release does four things:
- It fixes some important bugs discovered in Snowplow 0.9.5, related to our new shredding functionality
- It introduces new JSON-based configurations for Snowplow's existing enrichments
- It extends our geo-IP lookup enrichment to support all five of MaxMind's commercial databases
- It extends our referer-parsing enrichment to support a user-configurable list of internal domains
Java Tracker
- Bumped git submodule to 0.4.0 (#892)
EmrEtlRunner
- Bumped to 0.9.0
- Passed etl_tstamp into Hadoop Enrich as an argument (#396)
- Removed enrichment-specific code (#811)
- Removed enrichment-specific parameters from config.yml.sample (#809)
- Replaced enrichment-specific arguments from EmrEtlRunner (#808)
- Removed %3D code following Scalding upgrade (#849)
- Fixed contract on partition_by_run (#894)
- Updated Bash script to support enrichments path (#916)
StorageLoader
- Bumped to 0.3.1
- Now looking in eu-west-1 region for s3://snowplow-hosted-assets (#895)
- Updated combined Bash script to support enrichments path (#917)
Scala Hadoop Enrich
- Bumped to 0.6.0
- Bumped Scala to 2.10.4 (#912)
- Bumped Scalding to 0.11.1 (#911)
- Bumped Hadoop to 1.2.1 (#913)
- Bumped to Scala Common Enrich 0.5.0 (#788)
- Passed etl_tstamp into Scala Common Enrich (#817)
- Removed event_vendor and ue_name and renamed ue_properties to unstruct_event (#835)
- Removed %3D handling for compatibility with old Scalding Args (#850)
- Added ability to download additional MaxMind databases (#885)
- Added runHadoop and Tool.main tests (#914)
Scala Common Enrich
- Bumped to 0.5.0
- Bumped user-agent-utils version, thanks @pkallos! (#662)
- Bumped referer-parser to 0.2.2 (#864)
- Bumped httpclient to 4.3.3 (#897)
- Bumped scala-maxmind-geoip to scala-maxmind-iplookups 0.1.0 (#882)
- Stored etl_tstamp in new field in CanonicalOutput (#818)
- Removed event_vendor and ue_name and renamed ue_properties to unstruct_event (#836)
- Made referer parsing configurable with list of internal domains (#857)
- Migrated configurable enrichments to new EnrichmentRegistry (#858)
- Added validation of enrichments JSON (#807)
- Replaced "anon_ip_quartets" with "anon_ip_octets" everywhere (#547)
- Added ability to extract event_id from querystring (#723)
- Extracted CanonicalInput's userId as network_userid, thanks @pkallos! (#855)
- Added MaxMind region_name field (#873)
- Added IP -> ISP lookup (#861)
- Added IP -> organization lookup (#887)
- Added IP -> domain lookup (#886)
- Added IP -> net speed lookup (#889)
- Added validation for transaction ID (#428)
- Renamed Tests to Specs for consistency (#618)
Scala Hadoop Shred
- Bumped to 0.2.0
- Bumped to Scala Common Enrich 0.5.0 (#918)
- Trailing empty fields no longer cause shredding for that row to fail (#921)
- Updated column offsets for enriched events TSV (#915)
Redshift
- Bumped table-def to 0.4.0
- Migration script added for 0.3.0 to 0.4.0
- Added etl_tstamp to atomic.events (#819)
- Removed event_vendor and ue_name and renamed ue_properties to unstruct_event (#834)
- Added new MaxMind fields (#871)
- Applied runlength encoding to all fields keyed off IP address (#883)
- Migration script added for 0.3.0 to 0.4.0 (#838)