Updates Snowplow to run on the new 4.x series of Elastic MapReduce releases
Documentation
Common
- Made optionality of Lingual and HBase in config.yml clearer (#2206)
- Fixed OpenJDK build in Travis CI (#2447)
Scala Hadoop Enrich
- Bumped to 1.6.0
- Bumped Scala Common Enrich to 0.21.0 (#2442)
Scala Common Enrich
- Bumped to 0.21.0
- Fixed exception for invalid API key in currency conversion (#2441)
- Fixed exception on same currency conversion (#2437)
- Switched from javax.script to org.mozilla.javascript for JavaScriptEnrichment (#2453)
Scala Hadoop Shred
- Bumped to 0.8.0
- Bumped Iglu Scala Client to 0.3.2 (#2319)
EmrEtlRunner
- Bumped to 0.21.0
- Attached monitoring tags to jobflow (#425)
- Now throwing exception if processing thrift with --skip s3distcp or AMI 2.x.x (#1648)
- Added bootstrap action to prepare AMI >= 3.8.0 (#2320)
- Bumped Elasticity to 6.0.7 (#2400)
- Added support for Amazon EMR 4.x.x series (#1926)
- Prevented bad CLI options from throwing stack trace (#1930)
- Made error for nonempty processing bucket collector-agnostic (#1961)
- Bumped Ruby Tracker to 0.5.2 (#2143)
- Improved retry logic for EMR bootstrap timeouts (#2150)
- Excluded previously-built executables from the build (#2163)
- Added support for additional_info in EMR section of configuration (#2211)
- Added Elasticsearch stage to help message (#2323)
- Updated hadoop_enrich version in config.yml.sample to 1.6.0 (#2459)
- Updated hadoop_shred version in config.yml.sample to 0.8.0 (#2370)
- Removed snowplow-emr-etl-runner.sh (#2445)
StorageLoader
- Bumped to 0.7.0
- Added support for supplying config file as Base64-encoded string (#2227)
- Added ability to retrieve AWS credentials from EC2 role (#2226)
- Excluded previously-built executables from the build (#2164)
- Started printing stack trace for failures not caused by bad configuration (#2160)
- Bumped Ruby Tracker to 0.5.2 (#2144)
- Moved ANALYZE statements after VACUUM statements (#1361)
- Added resolver config option to snowplow-runner-and-loader.sh (#2170)
- Updated snowplow-runner-and-loader.sh to use JRuby binaries (#2233)
- Removed snowplow-storage-loader.sh (#2444)
- Wrote JSON Path file for com.optimizely/visitor_dimension event (#2436)
- Wrote JSON Path file for com.optimizely/visitor_audience event (#2435)
- Wrote JSON Path file for com.optimizely/visitor event (#2434)
- Wrote JSON Path file for com.optimizely/variation event (#2433)
- Wrote JSON Path file for com.optimizely/state event (#2432)
- Wrote JSON Path file for com.optimizely/experiment event (#2431)
- Wrote JSON Path file for io.augur.snowplow/identity_lite (#1958)
Redshift
- Wrote Redshift DDL for com.optimizely/visitor_dimension event (#2430)
- Wrote Redshift DDL for com.optimizely/visitor_audience event (#2429)
- Wrote Redshift DDL for com.optimizely/visitor event (#2428)
- Wrote Redshift DDL for com.optimizely/variation event (#2427)
- Wrote Redshift DDL for com.optimizely/state event (#2426)
- Wrote Redshift DDL for com.optimizely/experiment event (#2425)
- Added Redshift DDL for io.augur.snowplow/identity_lite (#1957)