Ports the Snowplow batch pipeline to Spark.
Documentation
- Fix incorrect hyphen underlining for R88 (#3198)
Common
- Refactor CI/CD deploy scripts into one (#3100)
- Update CI/CD to deploy Spark Enrich (#3069)
- Refactor CI/CD is release tag scripts into one (#3101)
- Update CI/CD to deploy RDB Shredder (#3038)
- Fix travis build due to the changes to the precise image (#3210)
- Build local Scala Common Enrich before publishing Kinesis-related artifacts (#3220)
- Add Sonatype credentials to .travis.yml (#3217)
- Bump Scala to 2.11 in .travis.yml (#3227)
Scala Common Enrich
- Bump to 0.25.0 (#3089)
- Bump scala-iglu-client to 0.5.0 (#3092)
- Remove scala-util (#3054)
- Get rid of deprecated erasure method calls (#3008)
- Bump scalaz to 7.0.9 (#3055)
- Bump scalding-args to 0.13.0 (#3058)
- Bump specs2 to 2.3.13 (#3059)
- Bump scalaz-specs2 to 0.2 (#3060)
- Bump scala-forex to 0.5.0 (#3057)
- Bump sbt to 0.13.13 (closes #3056)
- Bump Scala to 2.11.11 (#3007)
- Add Scala 2.11 cross-building (#3061)
- Make EnrichedEvent Serializable (#3081)
- Fix failing WeatherEnrichmentSpec expectation (#3205)
- Remove ScalazArgs (#3209)
- Upgrade to Java 8 (#3212)
- Add CI/CD (#3216)
Spark Enrich
- Bump to 1.9.0 (#3072)
- Rename from Scala Hadoop Enrich (#3064)
- Change the package from hadoop to spark (#3076)
- Bump sbt-assembly to 0.14.3 (#3078)
- Bump SBT to 0.13.13 (#3065)
- Port from Scalding to Spark (#3067)
- Bump scala-common-enrich to 0.25 (#3096)
- Bump Scalaz to 7.0.9 (#3097)
- Bump iglu-scala-client to 0.5.0 (#3098)
- Bump specs2-core to 2.3.13 (#3099)
- Bump Scala version to 2.11 (#3070)
- Upgrade to Java 8 (#2381)
- Fix SqlQueryEnrichmentCfLinesSpec (#3224)
- Fix CurrencyConversionTransactionSpec (#3225)
- Run the unit tests systematically in Travis (#3228)
EmrEltRunner
- Bump to 0.25.0 (#3039)
- Update to run Spark Enrich instead of Scala Hadoop Enrich (#3066)
- Update to run RDB Shredder instead of Scala Hadoop Shred (#3033)
- Add ability to run Spark jobs (#641)
- Replace hadoop_shred in config.yml.sample with rdb_shredder (#3035)
- Bump elasticity version to 6.0.11 (#3053)
- Use the Scalding step provided by Elasticity (#3052)
- Replace hadoop_enrich in config.yml.sample with spark_enrich (#3068)
- Bump AMI version in example config to 5.5.0 (#3207)
RDB Shredder
- Bump to 0.12.0 (#3042)
- Rename from Scala Hadoop Shred (#3031)
- Move from 3-enrich to 4-storage (#3032)
- Change the package to storage from enrich (#3036)
- Port from Scalding to Spark (#3034)
- Bump scala-common-enrich to 0.25 (#3091)
- Bump iglu-scala-client to 0.5.0 (#3090)
- Bump specs2-core to 2.3.13 (#3093)
- Bump Scala version to 2.11 (#3071)
- Upgrade to Java 8 (#3213)
- Run the unit tests systematically in Travis (#3229)