github GoogleCloudPlatform/DataflowTemplates 2023-10-17-01_RC00
Dataflow Templates 2023-10-17-01_RC00

latest releases: test_2024-09-20-00, 2024-09-19-00_RC00, 2024-09-18-00_RC00...
11 months ago

Release Week of 2023-10-17

Note: This release is being rolled out. It may not be in your region yet.

New Templates

*Cloud Spanner to Vector Search JSON format template

Improvements

*[JdbcToBigQuery] Allow overriding JdbcIO fetchSize
*Remove forked KafkaIO and classic KafkaToBigQuery template.
*[Spanner to Cloud Storage] Adding template to export Vector Embeddings in JSON format.
*Split Dataplex to its own module, avoid options conflicts
*[Spanner-to-BQ template] Add spannerProjectId parameter

Bug Fixes

*Keep a single SpannerUtils (common) to avoid name clashing
*Fix module settings after Dataplex became its own module
*Fix ResultSet closed errors in JDBCResourceManager.
*Do not duplicate already shaded resources in the Flex container

Contributors

@andreigurau
@bvolpato
@gauravpurohit06
@pranavbhandari24

Don't miss a new DataflowTemplates release

NewReleases is sending notifications on new releases.