After R118 beta release, R119 contains the first production versions of components emitting the new bad row format :
- Scala Common Enrich : Scala library that contains the validation and enrichments logic, used by Stream Enrich and Beam Enrich
- Stream Enrich : Scala application that performs the enrichments with either Kafka, NSQ or Kinesis as input and output
- Beam Enrich : Dataflow job that performs the enrichments on GCP with PubSub as input and ouput
- Scala Stream Collector : Scala application that receives the events sent by the trackers
This release also includes a new version of EmrEtlRunner with improved fault tolerance.
Beam Enrich gets extracted to its own repo.
Scala Common Enrich
- Use recent timestamp for unit tests in WeatherEnrichmentSpec (#4339)
- Change maxColumns to 140 in .scalafmt.conf (#4314)
- Add Snowplow Bintray to resolvers (#4325)
- Return capitalised Unknown value in YAUAA enrichment (#4114)
- Fix abandon assertions in PiiPseudonymizerEnrichmentSpec (#4322)
- Handle query string parameters that don't have a value in IgluAdapter (#4330)
- Bump snowplow-badrows to 1.0.0 (#4292)
- Fix EtlPipeline short-circuiting on a first bad row (#4320)
- Add validation for contexts added by enrichments (#3795)
- Bump to 1.1.0 (#4294)
Stream Enrich
- Add the possibility to override the DynamoDB endpoint (#3942)
- Use info generated by BuildInfo for Processor in Source (#4335)
- Remove sbt version from README (#4303)
- Bump Scala Common Enrich to 1.1.0 (#4295)
- Allow to download data from private S3 or GCS (#4269)
- Bump to 1.1.0 (#4296)
Scala Stream Collector
- Add Snowplow Bintray to resolvers (#4326)
- Publish Docker image for stdout via Travis (#4333)
- Fix config example (#4332)
- Fix incompatible jackson dependencies to enable CBOR (#4266)
- Bump to 1.0.1 (#4338)
EmrEtlRunner
- Switch AMI bootstrap scripts to HTTPS (#4256)
- Set ig.count (#4285)
- Catch and retry EMR connection exceptions (#4290)
- Bump to 0.37.0 (#4297)
Miscellaneous