apache/beam v2.43.0

Beam 2.43.0 release

on GitHub

We are happy to present the new 2.43.0 release of Beam.
This release includes both improvements and new functionality.
See the download page for this release.

For more information on changes in 2.43.0, check out the detailed release notes.

Highlights

• Python 3.10 support in Apache Beam (#21458).
• An initial implementation of a runner that allows us to run Beam pipelines on Dask. Try it out and give us feedback! (Python) (#18962).

I/Os

• Decreased TextSource CPU utilization by 2.3x (Java) (#23193).
• Fixed bug when using SpannerIO with RuntimeValueProvider options (Java) (#22146).
• Fixed issue for unicode rendering on WriteToBigQuery (#10785)
• Remove obsolete variants of BigQuery Read and Write, always using Beam-native variant
  (#23564 and #23559).
• Bumped google-cloud-spanner dependency version to 3.x for Python SDK (#21198).

New Features / Improvements

• Dataframe wrapper added in Go SDK via Cross-Language (with automatic expansion service). (Go) (#23384).
• Name all Java threads to aid in debugging (#23049).
• An initial implementation of a runner that allows us to run Beam pipelines on Dask. (Python) (#18962).
• Allow configuring GCP OAuth scopes via pipeline options. This unblocks usages of Beam IOs that require additional scopes.
  For example, this feature makes it possible to access Google Drive backed tables in BigQuery (#23290).
• An example for using Python RunInference from Java (#23290).

Breaking Changes

• CoGroupByKey transform in Python SDK has changed the output typehint. The typehint component representing grouped values changed from List to Iterable,
  which more accurately reflects the nature of the arbitrarily large output collection. #21556 Beam users may see an error on transforms downstream from CoGroupByKey. Users must change methods expecting a List to expect an Iterable going forward. See document for information and fixes.
• The PortableRunner for Spark assumes Spark 3 as default Spark major version unless configured otherwise using --spark_version.
  Spark 2 support is deprecated and will be removed soon (#23728).

Bugfixes

• Fixed Python cross-language JDBC IO Connector cannot read or write rows containing Numeric/Decimal type values (#19817).

List of Contributors

According to git shortlog, the following people contributed to the 2.43.0 release. Thank you to all contributors!

Ahmed Abualsaud
AlexZMLyu
Alexey Romanenko
Anand Inguva
Andrew Pilloud
Andy Ye
Arnout Engelen
Benjamin Gonzalez
Bharath Kumarasubramanian
BjornPrime
Brian Hulette
Bruno Volpato
Chamikara Jayalath
Colin Versteeg
Damon
Daniel Smilkov
Daniela Martín
Danny McCormick
Darkhan Nausharipov
David Huntsperger
Denis Pyshev
Dmitry Repin
Evan Galpin
Evgeny Antyshev
Fernando Morales
Geddy05
Harshit Mehrotra
Iñigo San Jose Visiers
Ismaël Mejía
Israel Herraiz
Jan Lukavský
Juta Staes
Kanishk Karanawat
Kenneth Knowles
KevinGG
Kiley Sok
Liam Miller-Cushon
Luke Cwik
Mc
Melissa Pashniak
Moritz Mack
Ning Kang
Pablo Estrada
Philippe Moussalli
Pranav Bhandari
Rebecca Szper
Reuven Lax
Ritesh Ghorse
Robert Bradshaw
Robert Burke
Ryan Thompson
Ryohei Nagao
Sam Rohde
Sam Whittle
Sanil Jain
Seunghwan Hong
Shane Hansen
Shubham Krishna
Shunsuke Otani
Steve Niemitz
Steven van Rossum
Svetak Sundhar
Thiago Nunes
Toran Sahu
Veronica Wasson
Vitaly Terentyev
Vladislav Chunikhin
Xinyu Liu
Yi Hu
Yixiao Shen
alexeyinkin
arne-alex
azhurkevich
bulat safiullin
bullet03
coldWater
dpcollins-google
egalpin
johnjcasey
liferoad
rvballada
shaojwu
tvalentyn

What's Changed

• Use cloudpickle for Java Python transforms. by @robertwb in #23073
• clean up comments and register functional DoFn in wordcount.go by @pcoet in #23057
• [Tour Of Beam][backend] integration tests and GA workflow by @eantyshev in #23032
• [Test] Decrease derby.locks.waitTimeout in jdbc unit test by @Abacn in #23019
• Auto-cancel old unit test Actions Runs by @Abacn in #23095
• Cross-langauge tests in github actions. by @robertwb in #23092
• Update CHANGES.md for 2.42.0 cut, and add 2.43.0 section by @lostluck in #23108
• remove "io/ioutil" package by @zaneli in #23001
• Add one NER example to use a spaCy model with RunInference by @liferoad in #23035
• Bump google.golang.org/api from 0.94.0 to 0.95.0 in /sdks by @dependabot in #23062
• Implement JsonUtils by @damondouglas in #22771
• Support models returning a dictionary of outputs by @damccorm in #23087
• [TPC-DS] Store metrics into BigQuery and InfluxDB by @aromanenko-dev in #22545
• [Website] Update from-spark page table content overflow by @bullet03 in #22915
• [Website] update homepage mobile styles by @bullet03 in #22810
• Use a ClassLoadingStrategy that is compatible with Java 17+ by @cushon in #23055
• [Website] Update case-studies logo images by @bullet03 in #22793
• [Website] Update ctas button container on homepage by @bullet03 in #22498
• [Website] fix code tags content overflow by @bullet03 in #22427
• Clean up Kafka Cluster and pubsub topic in rc validation script by @Abacn in #23021
• Fix assertions in the Spanner IO IT tests by @BjornPrime in #23098
• [Website] update shortcode languages by @bullet03 in #22275
• Use existing pickle_library flag in expansion service. by @robertwb in #23111
• Assert pipeline results in performance tests by @Abacn in #23027
• Consolidate Samza TranslationContext and PortableTranslationContext by @mynameborat in #23072
• Improvements to SchemaTransform implementations for BQ and Kafka by @pabloem in #23045
• [TPC-DS] Use common queries argument for Jenkins jobs by @aromanenko-dev in #23139
• pubsublite: Reduce commit logspam by @dpcollins-google in #22762
• [GitHub Actions] - Added documentation in ACTIONS.md by @dannymartinm in #23159
• Bump dataflow java fnapi container version to beam-master-20220830 by @Abacn in #23183
• [Issue#23071] Fix AfterProcessingTime for Python to behave like Java by @InigoSJ in #23100
• Don't depend on java 11 docker container for go test by @kileys in #23197
• Properly close Spark (streaming) context if Pipeline translation fails by @mosche in #23204
• Annotate stateful VR test in TestStreamTest with UsesStatefulParDo (related to #22472) by @mosche in #23202
• [Playground] [Backend] Datastore queries and mappers to get precompiled objects by @vchunikhin in #22868
• Allow and test pyarrow 8.x and 9.x by @TheNeuralBit in #22997
• (BQ Python) Pass project field from options or parameter when writing with dynamic destinations by @ahmedabu98 in #23011
• Update python-machine-learning.md by @AnandInguva in #23209
• Pin the version of cloudpickle to 2.1.x by @tvalentyn in #23120
• Add streaming test for Write API sink by @AlexZMLyu in #21903
• [Go SDK] Proto changes for timer param by @riteshghorse in #23216
• Bump github.com/testcontainers/testcontainers-go from 0.13.0 to 0.14.0 in /sdks by @dependabot in #23201
• Update to objsize to 0.5.2 which is under BSD-3 license (fixes #23096) by @lukecwik in #23211
• Exclude insignificant whitespace from cloud object by @csteegz in #23217
• Trying out property-based tests for Beam python coders by @pabloem in #22233
• Publish results of JMH benchmark runs (Java SDK) to InfluxDB (#22238). by @mosche in #23041
• Exclude protobuf 3.20.2 by @Abacn in #23226
• Fix IllegalStateException in StorageApiWriteUnshardedRecords error handling by @scwhittle in #23205
• Fix assignees check by @damccorm in #23229
• Update cibuildwheel by @AnandInguva in #23024
• Add section to docs on resource hints/RunInference by @damccorm in #23215
• (BQ Python) Perform job waits in finish_bundle to allow BQ streaming writes with large batch loads by @ahmedabu98 in #23012
• Update to newest version of CloudPickle. by @tvalentyn in #23223
• Resolve script parsing error when changing from bash to sh. by @lostluck in #23199
• Bump cloud.google.com/go/bigquery from 1.39.0 to 1.40.0 in /sdks by @dependabot in #23122
• Bump github.com/google/go-cmp from 0.5.8 to 0.5.9 in /sdks by @dependabot in #23123
• Update google-cloud-bigquery requirement from <3,>=1.6.0 to >=1.6.0,<4 in /sdks/python by @dependabot in #22378
• Optimize varint reading and writing for small ints. by @robertwb in #23192
• Pass namespace through RunInference transform by @AnandInguva in #23182
• [GitHub Actions] - INFRA scripts to implement GCP Self-hosted runners (Ubuntu&Windows) by @dannymartinm in #23158
• GA migration - Base actions to use for precommit and postcommit workflows by @benWize in #23109
• [Website] update site navigation #22902 by @bullet03 in #22968
• Test fix Kafka Performance test batch by @Abacn in #23191
• Revert "Exclude protobuf 3.20.2" by @Abacn in #23237
• Fix outdated code in python sdk install by @Abacn in #23231
• Bump up dataflow python container version to beam-master-20220914 by @Abacn in #23238
• Improve the performance of TextSource by reducing how many byte[]s are copied (fixes #23193) by @lukecwik in #23196
• Issue#21430 Avoid pruning DataframeTransforms by @KevinGG in #23069
• Bump cloud.google.com/go/bigquery from 1.40.0 to 1.41.0 in /sdks by @dependabot in #23247
• [Website] Correct spelling of structural by @shanemhansen in #23225
• TensorRT Initial commit by @azhurkevich in #22131
• Fix Kafka performance test sourceOption to match expected hash by @Abacn in #23274
• updated the pydoc for running a custom model on Beam by @liferoad in #23218
• Add drop_example flag to the RunInference and Model Handler by @AnandInguva in #23266
• Enable verbose output for RAT Precommit by @damccorm in #23279
• Bump google.golang.org/api from 0.95.0 to 0.96.0 in /sdks by @dependabot in #23246
• [Go SDK] Add timer coder support by @riteshghorse in #23222
• Fix wrong comment by @forsaken628 in #23272
• Revert "Trying out property-based tests for Beam python coders (#22233)" by @pabloem in #23265
• [Playground] [Backend] Cache component for playground examples by @vchunikhin in #22869
• [BEAM-13416] Introduce Schema provider for AWS model classes extending SdkPojo by @mosche in #16947
• [BEAM-14378] [CdapIO] SparkReceiverIO Read via SDF by @Amar3tto in #17828
• [Java SDK core] emit watermark from PeriodicSequence (#23301) by @je-ik in #23302
• Extend windmill.proto protocol used by google-cloud-dataflow-java runner by @arne-alex in #23256
• Allow longer Class-Path entries by @shanemhansen in #23269
• SpannerIO - Do not use .get() on ValueProvider during pipeline creation by @bvolpato in #23294
• Improved pipeline translation in SparkStructuredStreamingRunner by @mosche in #22446
• Use avro DataFileReader to read avro container files by @steveniemitz in #23214
• Change google_cloud_bigdataoss_version to 2.2.8. by @scwhittle in #23300
• Fix Nexmark default log level by @mosche in #23337
• Bump cloud.google.com/go/storage from 1.26.0 to 1.27.0 in /sdks by @dependabot in #23336
• [Go SDK] lint fixes to go by @riteshghorse in #23351
• Bump cloud.google.com/go/bigquery from 1.41.0 to 1.42.0 in /sdks by @dependabot in #23329
• Extract playground components by @alexeyinkin in #23253
• Bump Java FnApi Container version to beam-master-20220923 by @Abacn in #23352
• Bump org.nosphere.apache.rat from 0.7.0 to 0.8.0 by @dependabot in #23330
• GA Migration PreCommit and PostCommit Tables in CI.md by @fernando-wizeline in #23372
• Upgraded Flink on Dataproc support from Interacitve Beam by @KevinGG in #23271
• Stack Trace Decoration for Beam Samza Runner by @Sanil15 in #23221
• [#22478]: Add read_time support to Google Firestore connector by @yixiaoshen in #22966
• Changes CoGroupByKey typehint from List to Iterable by @ryanthompson591 in #22984
• Fix TextSource incorrect handling in channels that return short reads. by @lukecwik in #23376
• Add a tensorflow example to the run_inference_basic notebook by @ryanthompson591 in #23173
• RunInference Benchmarks UI by @AnandInguva in #23371
• set upper bound on google-cloud-profiler by @AnandInguva in #23354
• Make identifier of Date/Time/DateTime to be a public static field by @gabrywu in #23370
• Add ISSUE#23071 to CHANGES.md by @InigoSJ in #23297
• Pin objsize version to avoid regression in 0.6.0 by @lukecwik in #23396
• [Tour Of Beam] API adjustments by @eantyshev in #23349
• Adds support in Samza Runner to run DoFn.processElement in parallel inside Samza tasks by @xinyuiscool in #23313
• Regenerate Go Protos by @lostluck in #23408
• Support google-cloud-spanner v3 and fixes broken unit tests by @Abacn in #23365
• Add relevant docs to Cloud Profiler exceptions. by @rohdesamuel in #23404
• Update state cache to not fail when measuring object sizes. by @lukecwik in #23391
• Small pytorch notebook bug fix by @damccorm in #23407
• PubsubIO - Improve limit validations to consider attributes by @bvolpato in #23023
• Example of Online Clustering by @shub-kris in #23289
• Add documentation link to the interactive environment by @rohdesamuel in #23409
• Bump google.golang.org/api from 0.97.0 to 0.98.0 in /sdks by @dependabot in #23394
• Increase Go Dataflow Postcommit timeout to 5h by @lostluck in #23423
• [Playground] [Backend] Updating endpoints for playground examples by @vchunikhin in #22870
• Send JavaScript messages to Playground iframes when switching the language in docs (#22361) by @alexeyinkin in #22960
• [Playground] [Backend] Adding SDK to the example response by @vchunikhin in #22871
• [Playground] [Backend] Removing the code related to the Cloud Storage by @vchunikhin in #22872
• Expand Python cross language docs. by @robertwb in #23047
• [BEAM-10785] Change RowAsDictJsonCoder to not ensure ASCII while encoding by @harrydrippin in #22312
• Update Python katas to latest version of EduTools and Beam 2.41 by @iht in #23180
• RunInference Benchmarks UI by @AnandInguva in #23426
• Relax pip check in setup.py to allow installation via other package managers (e.g. poetry) by @dsmilkov in #23326
• replaced tabs with spaces in inference/readme.md file by @ryanthompson591 in #23446
• [Playground] [Backend] Adding the tags field to the example response by @vchunikhin in #22882
• [Playground] [Backend] Edited the function for getting executable name by @vchunikhin in #22957
• Batch encoding and decoding of schema data. by @robertwb in #23194
• Fix type inference for set/delete attr. by @robertwb in #23242
• Support VR test including TestStream for Spark runner in streaming mode by @mosche in #22620
• Add cron job to trigger Java JMH micro-benchmarks weekly by @mosche in #23388
• JdbcIO fetchSize can be set to Integer.MIN_VALUE by @mcGIh in #23444
• [Tour Of Beam] return taskSnippetId/solutionSnippedId by @eantyshev in #23419
• Beam 21465 add requires stable input to KafkaIO commit in finalize by @johnjcasey in #23230
• [Website] Add new Java quickstart by @VeronicaWasson in #22747
• Fix JdbcIOIT, which seems to have never worked by @pabloem in #21796
• Require time-bound flag for non-UW streaming Python jobs for new SDKs. by @robertwb in #23468
• Support DECIMAL logical type in python SDK by @Abacn in #23014
• [Website] fix navbar footer overlap by @bullet03 in #23369
• AI/ML pipelines master page documentation by @Juta in #23443
• [Minor] Fix go fmt error by @Abacn in #23474
• Revert "Add drop_example flag to the RunInference and Model Handler" by @AnandInguva in #23392
• Prep sidepanel 3.0.0 release by @KevinGG in #23469
• Documented supported PyTorch versions by @BjornPrime in #22974
• [Go SDK] Add fake impulse for inputs in Xlang Transform by @riteshghorse in #23383
• GA Migration Adding Removal of /.m2/settings.xml by @fernando-wizeline in #23481
• Bump google-cloud-spanner version for py containers by @Abacn in #23480
• Fix validation of measurement name in InfluxDBPublisher by @mosche in #23496
• Ensure Java JMH benchmark tasks run sequentially by @mosche in #23495
• [Tour Of Beam] export group.id in API by @eantyshev in #23445
• Give issue tagger permission to write issues by @damccorm in #23485
• Update UID of the RunInference Benchmarks by @AnandInguva in #23486
• Improve error message in GcsUtil by @pranavbhandari24 in #23482
• Add more typescript examples to the programming guide. by @robertwb in #23058
• opt in for schema update. addresses #23504 by @reuvenlax in #23505
• fix: only report backlog bytes on data records by @thiagotnunes in #23493
• Fix broken link in online clustering documentation by @shub-kris in #23516
• Grant actions using GITHUB_TOKEN the appropriate permission set by @damccorm in #23521
• Fix failing Py37 BQ file loads test by @ahmedabu98 in #23334
• [Website] update links to https by @raboof in #23523
• Support custom avro DatumReader when reading from BigQuery by @kkdoon in #22718
• Rename 'clean' Gradle task that required Flutter and has been breaking nightly build by @alexeyinkin in #23515
• Model handler unit test by @ryanthompson591 in #23506
• Content/multi model pipelines by @PhilippeMoussalli in #23498
• [Tour of Beam][Frontend] Content Tree and SDK models (#23316) by @nausharipov in #23417
• Fix bug where astype(CategoricalDtype) is rejected by @TheNeuralBit in #23513
• fix typo - s/befrehand/beforehand by @toransahu in #23519
• Bump actions/stale from 5 to 6 by @dependabot in #23331
• Correctly detect retryable TransientKinesisExceptions (fixes #23517) by @mosche in #23518
• Fixing right nav on Get Started page by @rszper in #23543
• Bump google.golang.org/grpc from 1.49.0 to 1.50.0 in /sdks by @dependabot in #23533
• Fix small error message typo by @damccorm in #23544
• update bom to the latest one. by @reuvenlax in #23547
• [Website] Update Hazelcast Jet runner by @bullet03 in #23040
• [Website] fix right nav-menu and videos overlap by @bullet03 in #23500
• [Website] update python-dependencies.md link by @bullet03 in #23479
• Support named databases in Firestore connector. Fix and enable Firestore IT test by @yixiaoshen in #23322
• [fixes #23000] Update the Python SDK harness state cache to be a loading cache by @lukecwik in #23046
• Fix permission for Build python wheel branch_repo_nightly step by @Abacn in #23563
• [Playground] complexity indicator by @Malarg in #23477
• Reolling forward property-based tests for coders by @pabloem in #23425
• Updated README for jupyterlab-sidepanel by @KevinGG in #23565
• Downgrade Scala version in Spark job-server to prevent Scala serialization bug by @mosche in #23522
• [Website] fix distribution example in golang guide by @ryoheinagao in #23567
• BEAM-13592 Add getOrderingKey in o.a.b.sdk.io.gcp.pubsub.PubsubMessage by @gemelen in #22216
• Add database role to SpannerConfig for role-based access control. by @hmehrotra in #23377
• Add a multi-process shared utility. by @robertwb in #23473
• Replace website references to deprecated aws / kinesis modules with more recent aws2 module. by @mosche in #23571
• Remove obsolete and deprecated bigquery native read. by @robertwb in #23557
• Feature/name all java threads by @Geddy05 in #23387
• [Go SDK] Don't construct plans in lock section. by @lostluck in #23583
• Remove obsolete and deprecated bigquery native write. #23557 by @robertwb in #23558
• Increase Python PostCommit timeout. by @tvalentyn in #23595
• [website][adhoc] Fix spellcheck errors and typos by @aromanenko-dev in #23575
• Migrate GcsOptions#getExecutorService to an unbounded ScheduledExecutorService by @lukecwik in #23545
• (BQ Java) Explicitly set coder for multi-partition batch load writes by @ahmedabu98 in #23602
• Fix typo in bootstrap_beam_venv.py by @sjvanrossum in #23574
• Bump github.com/spf13/cobra from 1.5.0 to 1.6.0 in /sdks by @dependabot in #23591
• [Playground][Tour Of Beam] Datastore entities split by origin by @eantyshev in #23088
• Use writer schema only for BigQueryIO Read API by @kkdoon in #23594
• [Go SDK]: SingleFlight bundle descriptor requests by @lostluck in #23589
• Extend a timeout to create a bt cluster. by @tvalentyn in #23617
• Use new github output format by @damccorm in #23624
• Tour of Beam frontend state management (#23420) by @alexeyinkin in #23572
• Added content: The direct runner is not suited to production pipelines by @rszper in #23512
• Adding beam blog info to the Community page by @rszper in #23524
• Automatically open module/packages for Java 11+ by @kileys in #23165
• Update publish_release_notes to generate PR list by @pabloem in #23630
• Remove artificial timeout in FirestoreV1IT, Dataflow runner is very slow by @yixiaoshen in #23578
• Bump Legacy dataflow container image tag by @AnandInguva in #23625
• Align translation logging for Spark dataset runner with rdd runner by @mosche in #23623
• Fix SparkSessionFactory to not fail when using Spark master local[*] by @mosche in #23622
• Minor improvements to the tpcds gradle build for Spark by @mosche in #23621
• [Spark dataset runner] Add direct translation of Reshuffle and Reshuffle.ViaRandomKey transforms. by @mosche in #23638
• Make GCP OAuth scopes configurable via pipeline options. by @lukecwik in #23644
• Update BQIO to a single scheduled executor service reduce threads by @johnjcasey in #23234
• [GitHub Actions] - Verify Release Build Workflow by @dannymartinm in #23390
• Blog post for Hop web in Google Cloud by @iht in #23652
• [Playground] Examples CI by @eantyshev in #23476
• [Tour Of Beam] README update by @eantyshev in #23318
• [Jenkins,Spark] Stop running Nexmark suite for deprecated Spark 2 runners by @mosche in #23639
• Bump google.golang.org/api from 0.98.0 to 0.99.0 in /sdks by @dependabot in #23655
• Fix beam_PerformanceTests_PubsubIOIT_Python_Streaming by @Abacn in #23607
• 2.42.0 Release Blog Post by @lostluck in #23406
• Docs for state in go by @damccorm in #22965
• Adds a Java RunInference example by @chamikaramj in #23619
• Fixes #23627: Speed up website checks by @pabloem in #23673
• Suppress a FloatingPointLiteralPrecision error by @cushon in #23667
• Improved test coverage and fix the implementation of Inject and CoGBK by @forsaken628 in #23307
• Fix python log_level_overrides cannot be used on flink and other portable runners by @Abacn in #23668
• Better error for disabling runner v2 with cross language pipelines. by @robertwb in #23669
• Update scopes to match the existing set and update test to clear credentials during failure by @lukecwik in #23677
• [TPC-DS] Use "nonpartitioned" input for Jenkins jobs by @aromanenko-dev in #23665
• [GitHub Actions] - Run RC Validations Workflow by @dannymartinm in #23531
• Add workflow to update milestone on issue close by @damccorm in #23629
• add website page about data processing for ML by @Juta in #23552
• [Go SDK] Dataframe API wrapper by @riteshghorse in #23450
• [Go SDK]: Adds Automated Python Expansion Service by @riteshghorse in #23582
• Include CombineFn's in all by @tvalentyn in #23685
• Bump google.golang.org/grpc from 1.50.0 to 1.50.1 in /sdks by @dependabot in #23654
• [Playground][Frontend] Tags filter for Examples Catalog (#22074) by @nausharipov in #23532
• [Go SDK] Extract output coders in expandCrossLanguage by @riteshghorse in #23641
• Python 3.10 support by @AnandInguva in #23587
• Fixes #22192: Avoids nullpointer error. Preserves previous behavior. by @pabloem in #23679
• Migrate nexmark to common config for cron jobs by @apilloud in #23609
• Deflaking tests for BQ row insertions. These tests were flaky due to … by @pabloem in #23678
• [23286] Run beam_PerformanceTests_InfluxDbIO_IT on beam-perf node by @kileys in #23680
• [23697] Add java 11 home to jenkins test by @kileys in #23708
• [Go SDK]: Enable automatic python expansion service by @riteshghorse in #23699
• Remove numpy C API dep from public declarations. by @robertwb in #23649
• [Go SDK] add expansion service option for extra packages by @riteshghorse in #23712
• Downgrade container cryptography version to avoid yanked version by @Abacn in #23711
• Update portable runner test timeout by @ryanthompson591 in #23696
• Vortex multiplexing streams by @reuvenlax in #23510
• Io jms fix ack message checkpoint by @rvballada in #22932
• [Playground] Examples CD by @eantyshev in #23664
• Keep Spark version in a single place only (BeamModulePlugin) by @mosche in #23603
• Update release instructions in Python 3.10 by @AnandInguva in #23702
• Move Tensorflow Documentation by @ryanthompson591 in #23729
• Bump golang.org/x/text from 0.3.7 to 0.4.0 in /sdks by @dependabot in #23686
• [ToB] [Frontend] Unit Content markdown styles (#23592) by @nausharipov in #23662
• Add reopen issue command by @damccorm in #23733
• Add example of real time Anomaly Detection using RunInference by @shub-kris in #23497
• Support TIMESTAMP type in BigQueryIO with BEAM_ROW output type, and in read_gbq by @svetakvsundhar in #23070
• Add PytorchBatchConverter by @yeandy in #23296
• Pin version to grpcio in build-requirements.txt by @AnandInguva in #23735
• Bump up python container versions. by @robertwb in #23716
• Reduce log flood in Python PostCommit flink task by @Abacn in #23635
• Speed up check on website links by @pabloem in #23737
• Remove yeandy from reviewers by @yeandy in #23753
• tests: fixes SpannerIO unavailable retry test by @thiagotnunes in #23748
• Revert bigdataoss version upgrade by @johnjcasey in #23727
• Updating Python dependencies for the 2.43.0 release by @chamikaramj in #23743
• [cherry-pick][release-2.43.0] Update GcsIO initialization to support converting input parameters to PipelineOptions for authentication (#23766) by @lukecwik in #23770
• Adds instructions for running the Multi-language Java quickstart from… by @chamikaramj in #23772
• Avoid pickling unstable reference to moved proto classes. (#23739) by @chamikaramj in #23798
• [release-2.43.0][cherry-pick] Allow MoreFutures.allAsList/allAsListWithExceptions to have the passed in list to be mutated (#23811) by @lukecwik in #23818
• [cherry-pick][release-2.43.0] Avoid Circular imports related to bigquery_schema_tools by @svetakvsundhar in #23784
• Cherry pick/revert gax executor by @johnjcasey in #23854
• Adds a dependency to Python Multi-language library to the GCP Bom examples arche-type by @chamikaramj in #23840
• [release-2.43.0] Revert accidental push by @kileys in #23911
• Merge pull request #23954: Emit job ids via side output in TriggerFileLoads process to keep beam.Flatten() happy for Spark and Flink runners by @chamikaramj in #23967
• [cherry-pick][Go SDK] Fix Dataframe wrapper issue by @riteshghorse in #24070
• cherry pick #24041: fixing BigQuery Query retry vulnerability by @ahmedabu98 in #24095
• Update staging of Python wheels (#24114) by @chamikaramj in #24127

New Contributors

• @AlexZMLyu made their first contribution in #21903
• @csteegz made their first contribution in #23217
• @shanemhansen made their first contribution in #23225
• @azhurkevich made their first contribution in #22131
• @forsaken628 made their first contribution in #23272
• @gabrywu made their first contribution in #23370
• @harrydrippin made their first contribution in #22312
• @dsmilkov made their first contribution in #23326
• @mcGIh made their first contribution in #23444
• @VeronicaWasson made their first contribution in #22747
• @raboof made their first contribution in #23523
• @toransahu made their first contribution in #23519
• @Malarg made their first contribution in #23477
• @ryoheinagao made their first contribution in #23567
• @gemelen made their first contribution in #22216
• @hmehrotra made their first contribution in #23377
• @Geddy05 made their first contribution in #23387

Full Changelog: v2.42.0...v2.43.0