github GoogleCloudPlatform/DataflowTemplates 2023-03-14-00_RC00
Dataflow Templates 2023-03-14-00_RC00

latest releases: 2024-09-26-00_RC00, test_2024-09-20-00, 2024-09-19-00_RC00...
18 months ago

Release Week of 2023-03-14

Note: This release is in the process of rolling out. It may not be in your region yet.

New Templates

  • Added MQTT To Pub/Sub Template.
  • Added Kinesis to Pub/Sub Flex template.
    Note: both templates are not included in the list of Google-provided templates yet. They're available for building from source only.

Improvements

  • [BigQueryToElasticsearch] Template now supports UDFs.
  • [TextIOToBigQuery] Templates now supports RECORD type in BigQuery schema definition.
  • [DatastreamToSpanner] Introduced a flag roundJsonDecimals, which if set to true, will run PARSE_JSON() with wide_number_mode=>round on JSON data (default false).
  • [PubsubToAvro] Template now accepts dash in outputFileNamePrefix.
  • Security fixes: mysql-connector-java version upgraded.
  • Various unit test and integration test improvements, code base cleanup, documentation updates.

Bug Fixes

  • Fixed issue #582 (MongoDBtoBigQuery with UDF: ScriptObjectMirror cannot be cast to bson.Document error).

Contributors

@bvolpato @anish97IND @PoulamiR1994 @Polber @pabloem @Deep1998 @pranavbhandari24

Don't miss a new DataflowTemplates release

NewReleases is sending notifications on new releases.