github GoogleCloudPlatform/DataflowTemplates 2023-07-25-00_RC00
Dataflow Templates 2023-07-25-00_RC00

latest releases: 2024-11-19-00_RC00, 2024-11-12-00_RC00, 2024-11-06-01_RC00...
16 months ago

Release Week of 2023-07-25

Note: This release is being rolled out. It may not be in your region yet.

New Templates

  • PubsubToRedis Template
  • BigtableChangeStreamsToGcs Template

Improvements

  • [GooglecloudToNeo4j templates] Allow edge nodes to be merged while creating edges
  • [Security] Bump hadoop-version to 3.3.6
  • [Integration Tests] Add integration tests for CBT->BQ pipeline
  • [PubSubToDatadog] Allow minBatchCount=1 for HTTP retry test
  • [PubSubToDatadog] Send batches before exceeding 5MB payload limit
  • [DatastreamToSpanner] Refactored the DDL class files in the template.
  • [Annotations] Add new metadata parameters (enum, boolean, number) to annotations
  • [Integration Tests] Add timestamp to Spanner databases, to allow usage of static instance

Bug Fixes

  • [Integration Tests] Improve resource manager cleanup, do not double fail. Do not stage the same template twice. Increase integration test parallelism
  • [Integration Tests] Retry Spanner instance deletion when there are quota exceptions
  • Fix bugs introduced in tests by Foreign Key DELETE CASCADE feature

Contributors

Ashraf Hanafy
Florent Biville
@aksharau
@AlexeyKu
@bvolpato
@viragtripathi

Don't miss a new DataflowTemplates release

NewReleases is sending notifications on new releases.