GoogleCloudPlatform/DataflowTemplates 2023-04-04-00

Release Week of 2023-04-04

Note: This release is in the process of rolling out. It may not be in your region yet.

N/A

[UDF Tests] Validate that error messages are bubbled to the caller
[Integration Tests] Change artifactClient to gcsClient name (deprecation) and RMs to non-static
[Syndeo Template] Any to any framework for Syndeo Template
[Docs] Improve docs generation (with link to external docs), and classic templates coverage
[Integration Tests] Create property to force test runs with streaming engine (-DenableStreamingEngine)
[GCS-to-Splunk] Add the missing Splunk options, found in the Pubsub-to-Splunk template, but pulling the options from the Beam SDK
[Syndeo Template] Adding a pubsub read error metric and some unit test for it
[Flex Templates] Add goog-dataflow-provided-template-version label
[Performance Tests] Initial implementation KafkaIO peformance test
[Integration Tests] Add JDBC to BigQuery IT's. Add jdbc driver staging to IT's
[Jdbc-to-BigQuery] Add documentation about using a query that comes from GCS
[Docs] Add "Open in Google Cloud Shell" link to generated readme files, add UDF callout and links to readme from templates. Generate all template docs using latest Freemarker README-template
[Integration Tests] Create GCS to Elasticsearch test that uses JSON Schema

[DlpText-to-BigQuery] Fix container path
[Jdbc-To-BigQuery] Allow query to be fetched from a GCS path if it's too large
[Kafka-to-PubSub] Fix SslConsumerFactory to localize SSL files once per worker. Add StandardOpenOption.WRITE to open file for writing.
[Tests] Use seconds precision when creating a test prefix, to avoid conflicts with other tests.