snowplow/snowplow r75-long-legged-buzzard

Release 75 Long-Legged Buzzard

on GitHub

Add support for Urban Airship and SendGrid webhooks

• Blog post

Scala Hadoop Enrich

• Bumped to 1.5.0
• Bumped Scala Common Enrich to 0.20.0 (#2200)
• Added test for loading Urban Airship Connect ndjson files (#2168)
• Added test for SendGrid Adapter (#2194)

Scala Common Enrich

• Bumped to 0.20.0
• Added JsonLoader for Urban Airship, Mixpanel et al (#2210)
• Added Adapter to pre-process Urban Airship events (#2167)
• Abstracted Mandrill reformatParameters function into Adapter (#2171)
• Added Adapter to pre-process SendGrid events (#1161)

EmrEtlRunner

• Bumped to 0.20.0
• Updated hadoop_enrich version in config.yml.sample to 1.5.0 (#2282)
• Added raw s3 -> hdfs step with group by (#2253)
• Added directory flattening code (#2232)
• Added support for ndjson loader format (#2251)
• Improved test coverage of runner.rb (#2250)

Redshift

• Added Redshift DDL for a com.sendgrid/processed event (#2172)
• Added Redshift DDL for a com.sendgrid/dropped event (#2173)
• Added Redshift DDL for a com.sendgrid/delivered event (#2174)
• Added Redshift DDL for a com.sendgrid/deferred event (#2175)
• Added Redshift DDL for a com.sendgrid/bounce event (#2176)
• Added Redshift DDL for a com.sendgrid/open event (#2177)
• Added Redshift DDL for a com.sendgrid/click event (#2178)
• Added Redshift DDL for a com.sendgrid/spamreport event (#2179)
• Added Redshift DDL for a com.sendgrid/unsubscribe event (#2180)
• Added Redshift DDL for a com.sendgrid/group_unsubscribe event (#2181)
• Added Redshift DDL for a com.sendgrid/group_resubscribe event (#2182)
• Added Redshift DDL for com.urbanairship.connect/UNINSTALL event (#2283)
• Added Redshift DDL for com.urbanairship.connect/TAG_CHANGE event (#2284)
• Added Redshift DDL for com.urbanairship.connect/SEND event (#2285)
• Added Redshift DDL for com.urbanairship.connect/RICH_READ event (#2286)
• Added Redshift DDL for com.urbanairship.connect/RICH_DELIVERY event (#2287)
• Added Redshift DDL for com.urbanairship.connect/RICH_DELETE event (#2288)
• Added Redshift DDL for com.urbanairship.connect/REGION event (#2289)
• Added Redshift DDL for com.urbanairship.connect/PUSH_BODY event (#2290)
• Added Redshift DDL for com.urbanairship.connect/OPEN event (#2291)
• Added Redshift DDL for com.urbanairship.connect/LOCATION event (#2292)
• Added Redshift DDL for com.urbanairship.connect/IN_APP_MESSAGE_RESOLUTION event (#2293)
• Added Redshift DDL for com.urbanairship.connect/IN_APP_MESSAGE_EXPIRATION event (#2294)
• Added Redshift DDL for com.urbanairship.connect/IN_APP_MESSAGE_DISPLAY event (#2295)
• Added Redshift DDL for com.urbanairship.connect/FIRST_OPEN event (#2296)
• Added Redshift DDL for com.urbanairship.connect/CUSTOM event (#2297)
• Added Redshift DDL for com.urbanairship.connect/CLOSE event (#2298)

StorageLoader

• Added JSON Path file for com.sendgrid/processed event (#2183)
• Added JSON Path file for com.sendgrid/dropped event (#2184)
• Added JSON Path file for com.sendgrid/delivered event (#2185)
• Added JSON Path file for com.sendgrid/deferred event (#2186)
• Added JSON Path file for com.sendgrid/bounce event (#2187)
• Added JSON Path file for com.sendgrid/open event (#2188)
• Added JSON Path file for com.sendgrid/click event (#2189)
• Added JSON Path file for com.sendgrid/spamreport event (#2190)
• Added JSON Path file for com.sendgrid/unsubscribe event (#2191)
• Added JSON Path file for com.sendgrid/group_unsubscribe event (#2192)
• Added JSON Path file for com.sendgrid/group_resubscribe event (#2193)
• Added JSON Path file for com.urbanairship.connect/UNINSTALL event (#2299)
• Added JSON Path file for com.urbanairship.connect/TAG_CHANGE event (#2300)
• Added JSON Path file for com.urbanairship.connect/SEND event (#2301)
• Added JSON Path file for com.urbanairship.connect/RICH_READ event (#2302)
• Added JSON Path file for com.urbanairship.connect/RICH_DELIVERY event (#2303)
• Added JSON Path file for com.urbanairship.connect/RICH_DELETE event (#2304)
• Added JSON Path file for com.urbanairship.connect/REGION event (#2305)
• Added JSON Path file for com.urbanairship.connect/PUSH_BODY event (#2306)
• Added JSON Path file for com.urbanairship.connect/OPEN event (#2307)
• Added JSON Path file for com.urbanairship.connect/LOCATION event (#2308)
• Added JSON Path file for com.urbanairship.connect/IN_APP_MESSAGE_RESOLUTION event (#2309)
• Added JSON Path file for com.urbanairship.connect/IN_APP_MESSAGE_EXPIRATION event (#2310)
• Added JSON Path file for com.urbanairship.connect/IN_APP_MESSAGE_DISPLAY event (#2311)
• Added JSON Path file for com.urbanairship.connect/FIRST_OPEN event (#2312)
• Added JSON Path file for com.urbanairship.connect/CUSTOM event (#2313)
• Added JSON Path file for com.urbanairship.connect/CLOSE event (#2314)

Data modeling

• Removed events enriched from web-recalculate (#2275)
• Added cookie-to-user-id map to web-recalculate (#2274)